{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzing Amazon SageMaker Tuning results\n",
    "Amazon SageMaker Automatic Model Tuning features a convenient `analytics().dataframe()` method that allows to extract all tuning job iterations with their hyperparameters and performance as a pandas dataframe. This then allows flexible analysis of the result, as shown below.\n",
    "\n",
    "SageMaker Model Tuning is flexible and when used with custom code, allows optimization of arbitrary metrics over arbitrary parameters. Hence be aware that depending on your optimization problem, some or the function below may not apply or would require minor editing to be adapted to your problem\n",
    "\n",
    "This notebook provide code samples demonstrating the flexible analysis capabilities enabled by SageMaker\n",
    "\n",
    "More information can be found in related documentation:\n",
    "* SageMaker Python SDK Analytics functionalities https://sagemaker.readthedocs.io/en/stable/analytics.html\n",
    "* Blog Post https://aws.amazon.com/blogs/machine-learning/easily-monitor-and-visualize-metrics-while-training-models-on-amazon-sagemaker/\n",
    "* Official SageMaker Tuning Analytics demo: https://github.com/awslabs/amazon-sagemaker-examples/blob/master/hyperparameter_tuning/analyze_results/HPO_Analyze_TuningJob_Results.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Seaborn version 0.9.0 already 0.9.0 or later\n"
     ]
    }
   ],
   "source": [
    "%%sh\n",
    "# freeze the Seaborn version (seaborn is a python dataviz package https://seaborn.pydata.org/)\n",
    "\n",
    "REQUIRED_SEABORN=0.9.0\n",
    "SEABORN_VERSION=$(pip freeze | grep seaborn | cut -d '=' -f 3)\n",
    "if [[ \"$SEABORN_VERSION\" < \"$REQUIRED_SEABORN\" ]]; then\n",
    "    echo \"Seaborn version $SEABORN_VERSION found, upgrading to $REQUIRED_SEABORN\"\n",
    "    pip install --upgrade seaborn==0.9.0\n",
    "else\n",
    "    echo \"Seaborn version $SEABORN_VERSION already $REQUIRED_SEABORN or later\"\n",
    "fi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib\n",
    "import boto3\n",
    "from matplotlib import pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "from matplotlib import cm\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import sagemaker\n",
    "from sagemaker.analytics import TrainingJobAnalytics\n",
    "import scipy\n",
    "import seaborn as sns\n",
    "\n",
    "\n",
    "# instantiate a boto3 client for Cloudwatch\n",
    "client = boto3.client('logs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = (12, 7)\n",
    "plt.rcParams['font.size'] = 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "boto3 1.13.11\n",
      "matplotlib 3.0.3\n",
      "pandas 0.23.0\n",
      "sagemaker 1.56.1\n",
      "scipy 1.4.1\n",
      "sns 0.9.0\n"
     ]
    }
   ],
   "source": [
    "print(f'boto3 {boto3.__version__}')\n",
    "print(f'matplotlib {matplotlib.__version__}')\n",
    "print(f'pandas {pd.__version__}')\n",
    "print(f'sagemaker {sagemaker.__version__}')\n",
    "print(f'scipy {scipy.__version__}')\n",
    "print(f'sns {sns.__version__}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a couple functions for data visualization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def scatter(data, x, y, z, title, logx=False, logy=False, label=False,\n",
    "               alpha=0.3, size=150):\n",
    "    \"\"\"\n",
    "       HPO-dedicated scatterplot function\n",
    "       objective is the metric to be optimized. It is encoded both by plot size \n",
    "       and plot color. Label values are manually truncated.\n",
    "    \"\"\"\n",
    "    \n",
    "    temp = data[[x,y,z]].dropna().reset_index()\n",
    "    ax = temp.plot.scatter(x=x, y=y, logx=logx, logy=logy, c=z, colormap='plasma',\n",
    "                          s=size*scipy.stats.rankdata(temp[z]),  # dot size inversely proportional to rank\n",
    "                          alpha=alpha, colorbar=False)\n",
    "    ax.set_xlabel(x)\n",
    "    ax.set_ylabel(y)\n",
    "    ax.set_title(title)\n",
    "    \n",
    "    if label != False:\n",
    "        for i in range(len(temp)):\n",
    "            ax.text(temp[x][i], temp[y][i], '%s' % (int((1000*temp[label][i]))/1000),\n",
    "                    zorder=1, color='k')\n",
    "    return ax\n",
    "\n",
    "            \n",
    "            \n",
    "def surface_plot(x, y, z, xlabel, ylabel):\n",
    "    \"\"\"3D surface of z as a function of x and y.\n",
    "       x, y and z provided as numpy or pandas 1D-arrays\"\"\"\n",
    "    \n",
    "    fig = plt.figure()\n",
    "    ax = fig.gca(projection='3d')\n",
    "    \n",
    "    viz = ax.plot_trisurf(x, y, z, cmap=cm.plasma, linewidth=0.1, antialiased=True)\n",
    "    \n",
    "    ax.set_xlabel(xlabel)\n",
    "    ax.set_ylabel(ylabel)\n",
    "    ax.set_zlim(z.min(), z.max())\n",
    "    \n",
    "    fig.colorbar(viz, shrink=0.5, aspect=5)\n",
    "    \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bring SageMaker Tuning results\n",
    "\n",
    "<font color=\"brown\"> **Edit the cell below to collect information for any running or completed hyperparameter tuning job. You can check the available jobs in the [AWS console](https://eu-west-1.console.aws.amazon.com/sagemaker/home?region=eu-west-1#/hyper-tuning-jobs/)] (set the correct region). You can also browse them with the SDK of your choice.**\n",
    "</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "%store -r tuning_job_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Previous Tuning Job Name: tensorflow-training-200612-1046\n"
     ]
    }
   ],
   "source": [
    "print('Previous Tuning Job Name: {}'.format(tuning_job_name))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "jobname = tuning_job_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "for this tuner, objective is to Maximize\n"
     ]
    }
   ],
   "source": [
    "tuner = sagemaker.tuner.HyperparameterTuner.attach(jobname)\n",
    "\n",
    "# define a boolean that will be used to sort the data in the right order, depending on the tuning direction:\n",
    "if tuner.objective_type == 'Maximize':\n",
    "    is_ascending = True\n",
    "elif tuner.objective_type == 'Minimize':\n",
    "    is_ascending = False\n",
    "    \n",
    "print('for this tuner, objective is to ' + tuner.objective_type)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 iterations\n",
      "best job: tensorflow-training-200612-1046-001-d6feee4c\n",
      "   FinalObjectiveValue  TrainingElapsedTimeSeconds           TrainingEndTime  \\\n",
      "0                 0.25                      2069.0 2020-06-12 11:22:57+00:00   \n",
      "1                 0.25                      2139.0 2020-06-12 12:06:31+00:00   \n",
      "\n",
      "                                TrainingJobName TrainingJobStatus  \\\n",
      "0  tensorflow-training-200612-1046-001-d6feee4c         Completed   \n",
      "1  tensorflow-training-200612-1046-002-cc196201         Completed   \n",
      "\n",
      "          TrainingStartTime       epsilon  learning_rate  cumulative  \n",
      "0 2020-06-12 10:48:28+00:00  1.392559e-08       0.000048        0.25  \n",
      "1 2020-06-12 11:30:52+00:00  5.685460e-08       0.000018        0.25  \n"
     ]
    }
   ],
   "source": [
    "results = tuner.analytics().dataframe()\n",
    "\n",
    "# Only use completed training jobs\n",
    "results = results[results.TrainingJobStatus == \"Completed\"]\n",
    "num_iterations = results.shape[0]\n",
    "\n",
    "if (num_iterations >= 1):\n",
    "    \n",
    "    # sort\n",
    "    results = (results[~results.FinalObjectiveValue.isnull()]\n",
    "               .sort_values(by='TrainingStartTime', ascending=True).reset_index(drop=True))\n",
    "\n",
    "    # best job so far\n",
    "    if is_ascending:\n",
    "        results['cumulative'] = results['FinalObjectiveValue'].cummax()\n",
    "    else:\n",
    "        results['cumulative'] = results['FinalObjectiveValue'].cummin()\n",
    "\n",
    "    best_job = results.sort_values(by='FinalObjectiveValue',\n",
    "                                   ascending=not is_ascending)['TrainingJobName'].values[0]\n",
    "    print('{} iterations'.format(len(results)))\n",
    "    print('best job: {}'.format(best_job))\n",
    "\n",
    "    # show top 5 best iterations\n",
    "    print(results.sort_values(by='FinalObjectiveValue', ascending=not is_ascending).head(5))\n",
    "else:\n",
    "    print(f'Job {jobname} has no tuning results to analyze yet.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convergence speed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvQAAAG3CAYAAADb+MFJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XmYVNWd//H3V2RVQFQQBCJqxt0kmlZUMGJE4xL3BUwyjqOGmGiMJm6oiUQT13GLOuISjSYxYJSJ474kIypxCUadjDv+lChoBMUVVJDv748qOkXT3VQ31d1cfb+ep57qOufce7/FfVo/dfvUuZGZSJIkSSqmFTq6AEmSJEmtZ6CXJEmSCsxAL0mSJBWYgV6SJEkqMAO9JEmSVGAGekmSJKnADPSSJElSgRnoJUmSpAIz0EuSJEkFtmJHF1A0q6++eg4ZMqSjy5AkSdKn3GOPPTY7M/subZyBvoWGDBnC1KlTO7oMSZIkfcpFxPRqxjnlRpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQXmspWSJKmm3n33Xd544w3mz5/f0aVIy6XOnTvTr18/evXqVZP9GeglSVLNvPvuu/zjH/9g4MCBdO/enYjo6JKk5UpmMm/ePGbMmAFQk1DvlBtJklQzb7zxBgMHDqRHjx6GeakREUGPHj0YOHAgb7zxRk32aaCXJEk1M3/+fLp3797RZUjLve7du9dsWpqBXpIk1ZRX5qWlq+XviYFekiRJKjADvSRJUgPjxo0jIpZ4jBw5kgULFhARjB8/vk2OPWjQIE488cTF2jKTq6++mi233JKVVlqJXr16MWLECG699dYlth8+fDijR49u9hjTpk0jIrjzzjtrWvvDDz/MaaedtkT7KaecQv/+/Wt6rEceeYSI4Oabb260f+bMmXTq1Inzzjuv6n2OHj2a4cOH16rEdmOglyRJakTv3r156KGHFntcfPHFrLjiijz00EPss88+7VbLmDFj+M53vsM222zDzTffzIQJExg8eDC77757iwLrIoMHD+ahhx5i6623rmmdTQX6ww8/nNtvv72mxxo6dCjrrLMOEyZMaLT/hhtuIDMZNWpUTY+7PHLZSkmSpEasuOKKbLXVVo32NdXeFm688UauuuoqrrzySg477LD69l133ZV+/fpxwgknMHLkSL74xS9Wvc+uXbu263sYNGgQgwYNqvl+R48ezUUXXcTcuXPp0aPHYn0TJkxg2223bZPjLm+8Qi9JktQCjU25WTTN5de//jXrrrsuvXr1Ytddd2XmzJmLbXvcccex6aabsvLKKzNo0CD+9V//dalLF1500UWsv/76HHLIIUv0nXLKKfTo0YNLL710ib7LLruMtdZai+7du7P77rsvVktTU24uv/xyNtpoI7p27cqQIUMavfp/3333MWLECFZaaSVWWWUVtt9+e5588kmuuuoqjjnmGD755JPFpigtqnPRlJv33nuP7t27c/nlly+x780224yDDz64/vX06dMZNWoUffr0oUePHuyyyy688MIL9f0HHnggH3zwAbfccsti+3n55Zd55JFHOPDAA+vbfvnLX7LNNtvQp08fVlttNUaOHMkTTzyxRA2VTjzxxCU+EHz44YdEBFddddVi7ZdddhkbbrghXbt2Ze211+bCCy9sdt+1ZKCXJElqwoIFCxZ7ZGaTY6dMmcLll1/OBRdcwPjx45k6dSqHH374YmNmzZrFSSedxG233cYFF1zA888/zw477MDChQsb3efHH3/MI488wh577MEKKywZ2/r06cN2223H/fffv1j7Aw88wPjx47ngggu48sor+etf/8q+++7b7Hs988wz+f73v8++++7LbbfdxpgxYxg7duxiH1zuvfdeRo4cSffu3bnuuuv43e9+xzbbbMPMmTPZc889Ofroo+nUqdNiU5Qa6tmzJ7vuuis33HDDYu3PP/88TzzxRP38/9mzZzNs2DCmTZvGFVdcwcSJE3n77bfZcccd+eijjwDYZJNN2GSTTZaYdjNhwgRWXHFF9ttvv/q26dOnc8ghh3DTTTfx61//mr59+zJ8+HBeeeWVZv9dqnH66adz9NFHc8ABB3Dbbbdx6KGHcvzxxy8R+ttMZvpowePLX/5ySpKkxj399NMdXUJNnHrqqQks8bjnnnty/vz5CeRll11WP37YsGG5yiqr5Ntvv13fdu6552ZE5EcffdToMRYsWJAvv/xyAjllypT69oEDB+YJJ5yQmZmvvPJKAnnJJZc0WesRRxyRK6+88mK1dO7cOV955ZX6tvvuu6++/szMF154IYG84447MjNzzpw52b179/zZz3622L7Hjh2ba665Zi5cuDAzM+vq6nLo0KH1rxu64IILslOnTku0n3zyybnGGmvUv77hhhuyU6dO+frrr9e3nXbaabnaaqvl/PnzMzPzxBNPzNVXXz3nzJlTP2b27Nm58sor5/jx4+vbfv7zn2fXrl3znXfeqW/74he/mLvssktT/2S5YMGC/Pjjj3PIkCF59tln17ePGjUqhw0bVv/6hBNOyIEDBy627bx58xLIK6+8MjMz33zzzezatWueddZZi4077rjjcvDgwU3WkLn03xdgalaRT51DL0mS2tRPb3mKp2e+2yHH3mjNXpy6+8at2rZ3797ce++9i7Wtv/76TY4fOnQovXv3/uexN9qIzGTmzJkMGTIEgFtvvZWf//znPP3007z77j//TZ5//nm22WabVtXZmC222GKxqSLbbbcdq666Ko8++mj9NJhKU6ZMYd68eey///4sWLCgvn2HHXbgzDPPZObMmfTs2ZPHHnuMSy+9dJnXUN9tt93o1q0bN954I0cccQQAEydOZN9992XFFUvx9N577+VrX/saK6+8cn1NvXv3ZvPNN2fq1Kl85zvfAUrz6E8++WT+8Ic/cNBBB/Hcc8/x5JNP8qMf/WixY/7tb3/j5JNP5uGHH2bWrFn17c8///wyvZcHHniAjz76qNF/u3PPPZd//OMfrLHGGst0jKUx0EuSJDVixRVXpK6ubon2ytBWaZVVVlnsdZcuXYDSnGsorQCz1157sf/++3PSSSfRt29fFi5cyLBhw+rHNNSvXz86d+7M9OnTm6xz+vTpDBw4cIntGtvXa6+91ug+Zs+eDTT9geWVV16hf//+ZCYDBgxospZq9ejRgz322IOJEydyxBFH8NRTT/HUU08tNkVn9uzZTJ06ld/+9rdLbF95N+J11lmHoUOHMmHCBA466CAmTJhAt27d2GuvverHzJkzh5122om11lqLiy66iMGDB9OtWzcOOuigJv/tq7Xo327ddddttP+VV14x0EuSpGJr7RXyT5tJkyYxYMAArr/++vor3C+++GKz23Tp0oWhQ4dy6623cvbZZy9xZfztt9/m/vvvX2Jpxsa+aPvGG280GcZXXXVVAO644w5WX331Jfo32GADFi5cSEQ0+aGgpUaNGsXee+/NjBkzmDhxIv3792e77bZbrKbNNtuMk046aYlte/XqtdjrAw88kOOOO44333yTiRMn8vWvf52ePXvW9z/wwAO8/vrrPPTQQ/V/LYFS0G9Ot27d+Pjjjxdra7jNon+7u+++mz59+iyxjw033LDZY9SCgV6SJKkdzJs3jy5duiwWyhu7+tzQD37wA/bff3+uueaaJVa6OeOMM/jggw/qp60s8pe//IUZM2bUX7mfPHkyb731FltuuWWjx9hmm23o1q0br732GjvvvHOTtdTV1XHddddx+OGHNzrtpkuXLnzyyScsWLCgfupMU3beeWd69erF73//eyZOnMj++++/2Bd/d9hhB26++WY23XRTunbt2uy+DjjgAH74wx/yk5/8hGeeeYaf/exni/XPmzcPYLH9/OlPf+L1119vdr+DBg1i9uzZzJ49u/6Dzt13373YmOHDh9OlSxdef/11dtxxx2b311YM9JIkSe1gxx135JJLLuGHP/whu+22Gw8++GBVgX6//fbjsMMO4zvf+Q5/+9vf2G233Zg/fz4TJkzguuuu49xzz11iDfq+ffuy6667Mm7cOObOncvxxx/Plltu2ej8eYDVVluNH//4xxx55JG89NJLDB8+nIULF/Lcc89x//33c9NNNwFw9tlns9NOO7Hbbrvx7W9/mx49ejBlyhS23nprdtllFzbYYAMALrjgAkaMGEHv3r1Zb731Gj1m165d2WuvvTjnnHN47bXXuPrqqxfrP/bYY7n++uv56le/ypFHHsmaa67J66+/zuTJkxkxYgQHHHBA/dgBAwYwYsQILrvssvolQysNGzaM7t27c+ihh3LMMcfw8ssvc9pppy317rW77bYbnTt35uCDD+aoo47ihRdeWGK5zb59+3LyySfz3e9+l2nTpjF8+HAWLFjAc889x5///OclVvNpCy5bKUmS1A722GMPzjjjDCZOnMgee+zBlClTllg/vSlXXHEFl19+OVOmTGHPPffkgAMOYPr06dxyyy0ce+yxS4zfdtttGTNmDEcddRSHHXYYX/rSl5g0adIS4yqvsp900klcdtll3Hrrreyxxx584xvf4He/+x1f+cpX6sdsv/323H333bz77rt84xvfYNSoUTz44IP1fwnYfvvt+eEPf8gFF1zA0KFD+d73vtfs+xo9ejSvvfYagwcPXuJLwf369ePhhx/m85//PEcffTQ77bQTJ5xwAu+99x6bbrrpEvs68MADyUz23ntvunXrtljfoEGDmDhxIi+99BJ77LEHl156KVdffTVrrbVWs/UNGDCAG264gRdffJG99tqLG2+8sdEPYT/5yU+4+OKL+e///m923313vvnNbzJx4kS23XbbZvdfK5HNrKeqJdXV1eXUqVM7ugxJkpZLzzzzTLvMGdayefzxx9l888159NFH2WKLLTq6nM+spf2+RMRjmbnkN7MbcMqNJEnSZ8hf//pXfvGLX7Dyyis3uwynisNAL0mS9Bly8MEH8/7773PVVVctsVqMislAL0mS9Bnyv//7vx1dgmrML8VKkiRJBWaglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgBnpJkiSpwAz0kiRJUoEZ6CVJkgpoxIgR7Lfffi3a5vnnn2fcuHG8/fbbi7X/6le/IiJ4//33a1mi2omBXpIk6TPi+eef56c//ekSgX633XbjoYceokePHh1UmZaFd4qVJEn6jOvbty99+/bt6DLUSl6hlyRJasT999/P9ttvz8orr0zv3r0ZMWIEjz/+OOPGjWP11VdfYnxEcMkll9S/HjJkCMceeyxnnXUWAwYMoHfv3vzoRz8iM7n99tvZeOON6dmzJ3vttRdz5syp366p6S+L9teUZ599ltGjRzN48GB69OjBxhtvzIUXXsjChQsBuO+++9h9990BWHvttYkIhgwZ0ugx1157bY477rgljrH//vszfPjw+tdvvfUWY8aMYY011qBbt25ss802PPLII0v7p1WNeYVekiSpgfvuu48dd9yR7bffnmuvvZaVVlqJKVOmMGPGjBbtZ8KECWy55ZZcc801PPbYY5xyyiksXLiQ+++/n9NPP5158+Zx5JFHMnbsWMaPH79MNc+YMYP111+fb37zm/Ts2ZMnnniCU089lXnz5jF27Fg233xz/uM//oNjjz2WSZMmMWDAALp27drovg444AAmTpzIueeeW9/2/vvvc9ttt3HOOecA8NFHHzFy5Ejefvttzj33XPr168dll13GyJEjeeGFF+jfv/8yvR9Vz0AvSZLa1h0nwut/65hj998UdjmrxZuNHTuWL37xi9x1111EBAA777wzAFOnTq16P926deP3v/89nTp1Yuedd+bmm2/m4osv5oUXXmDttdcG4Mknn+Taa69d5kC/ww47sMMOOwCQmQwfPpy5c+dy5ZVXMnbsWHr16sX6668PwGabbVZ/db4xo0eP5pxzzuHhhx9mq622AuCWW27h448/Zv/99wfgN7/5Df/3f//HU089xb/8y78AMHLkSNZff33OO++8xT4MqG0Z6CVJkip88MEHPPLII1x00UX1Yb61RowYQadOnepff/7zn+ett96qD/OL2mbNmsXHH39Mly5dWn2sDz/8kDPPPJPf/va3/P3vf2f+/Pn1fQsWLGDFFauPfZttthnrrbceEydOrA/0EydOZLvttmONNdYA4N577+XLX/4ya6+9NgsWLKjfdrvttmvRhx4tOwO9JElqW624Qt6R5syZQ2YyYMCAZd7XKqusstjrLl26NNqWmcsc6E844QSuuuoqTj31VDbffHNWWWUVbr75Zn72s5/x4YcfsvLKK7dof6NGjeLqq6/m/PPP57333uPOO+/k4osvru+fPXs2Dz/8MJ07d15i23XXXbfV70MtZ6CXJEmq0KdPH1ZYYQVee+21Rvu7devGxx9/vFhb5Zdal1W3bt0AWnyM3//+93z/+9/n+OOPr2+77bbbWl3HqFGjOP3003nwwQd56aWXWLhwIfvss099/6qrrkpdXR2XXXbZEts2NTdfbcNAL0mSVGGllVZi6NChXHfddRx55JFLTLsZNGgQ7733HjNmzGDgwIEA3H333TU7/qBBgwB45plnGDZsGACPPPII7777brPbzZs3b7Eg/cknnzBhwoTFxiz6C8CHH3641Do23nhjNtlkEyZOnMhLL73EyJEjWW211er7d9hhB+6++24+97nP0a9fv+renNpEuwf6iNgIuBjYGngbuAr4aWZ+0sw2WwDfA7YF1gReAa4Hzs7MDxuM7QGcChwI9ANmAuMz85yKMV2BM4B/BVYC7gOOyMyXa/ImJUlSoZ111lmMHDmSXXbZhTFjxrDSSivx0EMPUVdXx84770z37t055JBD+NGPfsRLL720zF9orbTlllsycOBAjjrqKE4//XTeeustzjnnHHr16tXsdjvuuCOXXnopn//851l11VW59NJL+eijjxYbs+hLsZdffjmjR4+mR48ebLrppk3uc9SoUVx00UW88847XHnllYv1HXTQQYwfP54RI0Zw7LHHss466/Dmm2/y6KOP0r9/f4455phW/guopdp1HfqI6APcCySwJ3Aa8CPgp0vZdBSwLnA2sCtwKfBD4LcN9t8JuL2875OBnYGfN7K/XwAHA8cC+wGrA/dERLdWvC1JkvQp85WvfIV77rmHuXPn8q1vfYtRo0YxefJkBg0axOqrr85NN93Eq6++yl577cVvfvMbrr/++podu0uXLvzXf/0XK6ywAvvttx/nnXcel112GX369Gl2u4svvphtt92WI444gkMOOYRNNtmEsWPHLjZmrbXW4j/+4z+YNGkSw4YNq1+XvimjR49m9uzZrLDCCuy1116L9XXr1o3/+Z//Yccdd+TUU09lp5124gc/+AEvvPACW265ZevevFolMrP9DhYxFjgeWCsz3y23HQ+MA/ovamtku9Uzc3aDtjHA5cCQzJxebjscOBNYPzPfaGJfg4CXgUMy87py20DgJeB7mXlVc++hrq4u/ea2JEmNe+aZZ9hwww07ugypEJb2+xIRj2Vm3dL20953it0FuKtBcJ8AdAe2a2qjhmG+7PHy85oVbYcANzQV5st2Kj9Pqtj/DODBcn2SJElSYbR3oN8AeLayITP/Dswt97XE1sBC4EWAiOgCbAa8GhG/jYh5EfFORFwTEZWTzjYAXs3M9xvs75lW1CBJkiR1qPYO9H0ofRG2oTnlvqpERH/gFODXFVfjV6P0Jd/jKX3RdQ9K8+z3pPTF21bXEBFjImJqREydNWtWtWVKkiRJba5wy1aWr8TfALwPVH59etGaUnOA/TNzfnn8fODaiFg3M19szTEz8wrgCijNoW9t7ZIkSVKttfcV+jlA70ba+5T7mhWlhWCvAzYGds3Mym0WXXWfsijMl/2p/LxRLWqQJEnNa88FN6SiquXvSXsH+mdpME89IgYDPWgwt74JF1KaQrNnZjaciz8XmM4/r9TXH6L8vLCihsERsVKDcUvM75ckSS3TuXNn5s2b19FlSMu9efPm0blz55rsq70D/R3A1yKiZ0XbKGAeMLm5DctLXh4JfCszH2xi2K3AsPK0nEV2oBTm/1Z+vehWbntX7HtNSjetuqPK9yFJkhrRr18/ZsyYwdy5c71SLzUiM5k7dy4zZsyo2R1223sO/XjgKGBSRJwNrENpDfrzK5eyjIhpwOTMPLT8+huU7uz6K2BGRGxVsc8XM3PRN1XPBb4F3BQR/wkMpnQzqqvLq+mQma9GxC+BC8tTeGaVa5gO/KYt3rQkSZ8Vi+5mOnPmTObPn7+U0dJnU+fOnVljjTWWevffarVroM/MORGxA3AJcAulee8XUArUDevqVPF60drxB5cflf6dUtAnM6dHxMjyPicB7wLXAic22OYo4APgfErTfSYDB2bmh616Y5IkqV6vXr1qFlQkLV273in208A7xUqSJKk9LK93ipUkSZJUQwZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSqwdg/0EbFRRPwxIuZGxMyIOC0iOi1lmy0i4pqImFbe7rmIODUiujUYNy4ispHHzg3GNTbm4bZ4v5IkSVJbWrE9DxYRfYB7gaeBPYF1gfMofbA4pZlNR5XHng28AHwBOL38vG+Dse8AOzdoe6aRfZ4H3Fjx+r2q3oQkSZK0HGnXQA8cDnQH9snMd4F7IqIXMC4izim3NeaszJxd8fq+iPgQuDwi1srM6RV9CzKzmqvtL1c5TpIkSVputfeUm12AuxoE9wmUQv52TW3UIMwv8nj5ec3alSdJkiQVS3sH+g2AZysbMvPvwNxyX0tsDSwEXmzQvkpEzI6I+RHxeETs08T24yJiQXns1RGxaguPL0mSJHW49g70fYC3G2mfU+6rSkT0pzTn/teZ+UZF1zTgeGB/SnPrZwI3NRLqrwW+A3wVOAPYm9L0n0a/nBsRYyJiakRMnTVrVrVlSpIkSW2uvefQL7OI6ALcALwPHFPZl5m/aTD2FuDPwE+ASRXjDq4Ydn9EPAPcDuwO/KHhMTPzCuAKgLq6uqzF+5AkSZJqob2v0M8BejfS3qfc16yICOA6YGNg18xsdpvMTEpB/gtLWRrzTkofEDZfWg2SJEnS8qS9r9A/S4O58hExGOhBg7n1TbiQ0nKXO2ZmNeMBsvxoekBmlj4rND9OkiRJWt609xX6O4CvRUTPirZRwDxgcnMbRsRY4EjgW5n5YDUHK1/R3xd4MjM/aWbczsDKwGPV7FeSJElaXrT3FfrxwFHApIg4G1gHGAecX7mUZURMAyZn5qHl19+g9OXVXwEzImKrin2+mJmzyuMmAzdRutq/EvBtYCiwV8W+xwB1lG5wNZvSNJtTgEeB22r+jiVJkqQ21K6BPjPnRMQOwCXALZRWvLmAUqhvWFflnPedys8Hlx+V/p1S0IfSKjdHAwMoLWn5V2C3zLyjYvyLwL9RunLfC3id0rz8Hzd3FV+SJElaHkXpe6OqVl1dXU6dOrWjy5AkSdKnXEQ8lpl1SxvX3nPoJUmSJNWQgV6SJEkqMAO9JEmSVGAGekmSJKnAqg70EbFxRPw2Ip6LiA8iYrNy+2kR8bW2K1GSJElSU6oK9BGxE/A4MASYAHQHoty9kNLa8pIkSZLaWbVX6M8GfpOZw4DTG/T9FfhSTauSJEmSVJVqA/2GwPXlnxsuXP8OsFrNKpIkSZJUtWoD/Sxg7Sb6NgJeqU05kiRJklqi2kA/ETgtIraqaMuI+DxwAvC7mlcmSZIkaalWrHLcKcAmwBTg1XLbJGBN4E8sOa9ekiRJUjuoKtBn5ofAzuXlKXcAVgfeAv6YmXe0YX2SJEmSmlHtFXoAMvMu4K42qkWSJElSC1UV6CNivaWNycznl70cSZIkSS1R7RX6Z1lyucqGOi1jLZIkSZJaqNpAv2MjbX2ARXPqj6lZRZIkSZKqVu2XYv/YRNeNEXEGsBdwc82qkiRJklSVatehb84fgX1qsB9JkiRJLVSLQL8z8E4N9iNJkiSphapd5eb6Rpq7ABsAG1K68ZQkSZKkdlbtl2IHN9L2IfAX4KTM/O/alSRJkiSpWtV+KXbbti5EkiRJUsvVYg69JEmSpA7S5BX68nKU1crMPLkG9UiSJElqgeam3PxrC/aTgIFekiRJamdNBvrMbOyLsJIkSZKWI86hlyRJkgqs2mUrAYiIrYD1gG4N+zLziloVJUmSJKk61d5Yqi9wL7AppfnyUe7KimEGekmSJKmdVTvl5jxgLrA2pTC/DfB54KfANEp3jJUkSZLUzqqdcrM9cDTwSvn1wsz8f8BpEQHwC2CX2pcnSZIkqTnVXqFfBXgjMxcC7wJ9K/oeBIbXujBJkiRJS1dtoH8JGFD++WngwIq+3YC3a1mUJEmSpOpUG+jvBHYu/3wGMCoipkfEC5Sm4lzSFsVJkiRJal5Vc+gz8/iKn2+NiOHAPkB34J7MvKWN6pMkSZLUjCYDfURskZl/aawvMx8BHmmzqiRJkiRVpbkpN49ExAsRcVpEbNRuFUmSJEmqWnOB/gDgSeBHwN8i4smIOCEihrRHYZIkSZKWrslAn5k3ZuZ+wBrAvwF/B04DXoyIP0fEkRGxRjvVKUmSJKkRS13lJjO0Sk9PAAAeeUlEQVTfz8zfZObulML9GOB94ALg1Yi4JyL+vY3rlCRJktSIapetBCAz387MX2bmTsCawHjgq8BVbVGcJEmSpOZVtWxlpfIc+tHlxxco3Tn2v2palSRJkqSqVBXoI2JNYBSlEF8HfATcCvwUuD0zP2qzCiVJkiQ1qbl16FcH9qcU4ocBC4F7KH1B9g+Z+X67VChJkiSpSc1doX+N0hz7B4AjgBsz8812qUqSJElSVZoL9CcAEzNzRnsVI0mSJKllmgz0mXl+exYiSZIkqeVatGylJEmSpOWLgV6SJEkqMAO9JEmSVGAGekmSJKnAqg70EdE1Ir4bEb+MiLsj4l/K7aMiYsO2K1GSJElSU6q9U+x6lG4q1Rt4DBgB9Cx3bwvsBhzUBvVJkiRJaka1V+h/AfwdGAJ8DYiKvsnA8NqWJUmSJKkaVV2hp3QVfv/MfDsiOjXo+wcwoLZlSZIkSapGtVfoPwS6N9E3EHi7NuVIkiRJaolqA/09wEkR0buiLSOiK/B94PZqDxgRG0XEHyNibkTMjIjTGrnq33CbLSLimoiYVt7uuYg4NSK6NRg3LiKykcfODcZ1jYjzIuKNiPggIm6LiCHVvgdJkiRpeVHtlJvjgCnANErhPoGfABsDXYB9qtlJRPQB7gWeBvYE1gXOo/TB4pRmNh1VHns28ALwBeD08vO+Dca+A+zcoO2ZBq9/AewHHAPMAsYB90TEppn5YTXvRZIkSVoeVBXoM/OViPgi8ENgB+BFSvPmfw+cn5lvVnm8wylN3dknM9+lFKJ7AeMi4pxyW2POyszZFa/vi4gPgcsjYq3MnF7RtyAzH26qgIgYBBwKHJKZ15Xb/hd4CfgWcFWV70WSJEnqcFWvQ5+ZczLzx5m5TWaul5lbZebJLQjzALsAdzUI7hMohfztmjn27EaaHy8/r9mC4wPsVH6eVLH/GcCD5fokSZKkwqgq0EfEdRGxy9LmuldhA+DZyobM/Dswt9zXElsDCyn9taDSKhExOyLmR8TjEdFwOtAGwKuZ+X6D9mdaUYMkSZLUoaqdQ78hcBvwVkT8F/A74H8yM1t4vD40viLOnHJfVSKiP6U597/OzDcquqYBx1O6et8T+A5wU0Tsm5mLrsjXpIb29vB/fpuebzf8KoAkSZLa0nurbMhW37uyo8toVrVz6LeIiLWB0cABlOagvxERNwITM/OBNqxxMRHRBbgBeJ/Sl1or6/xNg7G3AH+m9AXeSbRSRIwBxgB87nOfa+1uJEmSpJqr9go9mfkScCZwZkSsT2nlmQOA70bEjMysJunOAXo30t6n3NesiAjgOkqr6wzLzGa3ycyMiEnA2RHRKTM/aU0NmXkFcAVAXV1dS/8qURPL+ydDSZIkdYyqvxRbKTOfA64GrgFep3RzqWo8S4N56hExGOhBg7n1TbiQ0nKXe2ZmNeOhtMRmZQh/FhgcESs1GLfE/H5JkiRpedeiQB8R/SPi+xHxIPAyMJbSTaV2anbDf7oD+FpE9KxoGwXMAyYv5dhjgSOBb2Xmg1XWG5TWqX+yfHUe4O7y894V49YEti3XJ0mSJBVGVVNuIuK7lKbXDAc+AG4GzgDuzswFLTjeeOAoYFJEnA2sQ+mmTudXLmUZEdOAyZl5aPn1N8rH+xUwIyK2qtjni5k5qzxuMnATpSvtKwHfBoYCey0anJmvRsQvgQvLgX/RjaWmA4vNwZckSZKWd9XOoT8XuBXYH7gjMz9qzcEyc05E7ABcAtxCabWZCygF6oZ1VS6RuegvAAeXH5X+nVLQh9IqN0dTuunVQuCvwG6Z2fDK+1GUPpicT2m6z2TgQO8SK0mSpKKJalaejIgemTm3HepZ7tXV1eXUqVM7ugxJkiR9ykXEY5lZt7RxVc2hN8xLkiRJy6cmp9xExBvA1zLz8YiYxeIrxSwhM/vVujhJkiRJzWtuDv2lwD8qfu6Q9dclSZIkNa3JQJ+ZP634eVy7VCNJkiSpRaqaQx8Rf4qIDZroWy8i/lTbsiRJkiRVo9obS40AejXR1wv4Sk2qkSRJktQiLblT7BJz6COiC/BV4PWaVSRJkiSpas2tcnMq8JPyywQeLt1YtVHn1rguSZIkSVVobpWb24HZQAC/AM4DXm4w5mPg2cx8oE2qkyRJktSs5la5+QvwF4CIeA+4NTPfbK/CJEmSJC1dtXPonwCGNtYREbtGxBdqV5IkSZKkalUb6C+giUAPbFHulyRJktTOqg30mwNTmuh7CNisNuVIkiRJaolqA30nYKUm+lYCutSmHEmSJEktUW2g/wswpom+McDU2pQjSZIkqSWaW7ay0jjg3oh4BLiW0o2kBgAHAV8EdmyT6iRJkiQ1q6pAn5n3R8ROwJnAxZTWpl8IPALs6Dr0kiRJUseo9go9mXkfsHVE9AD6AHMyc25bFSZJkiRp6aqdQw9ARASwGrAWpav0kiRJkjpQ1YE+Ir4HzACmAw8A65fbJ0XE0W1TniRJkqTmVBXoI+I44HzgSuCrLH51/j5gVM0rkyRJkrRU1c6hPwL4SWaeExGdGvQ9B6xX27IkSZIkVaPaKTf9gcea6FsIdKtNOZIkSZJaotpAPw3Yrom+rwBP16YcSZIkSS1R7ZSbC4H/jIiPgRvLbf0i4lDgh8C326I4SZIkSc2r9sZSV0VEH+AnwE/LzbcDc4FxmXl9G9UnSZIkqRktubHUuRExHtgaWB14C3goM99pq+IkSZIkNa/qQA+Qme8Bd7dRLZIkSZJaqMlAHxG7Ag9m5rvln5uTlK7YP5OZ79ayQEmSJElNa+4K/a3AVsCj5Z+TxW8o1ZgPIuKwzJxYo/okSZIkNaO5QL828FrFz0vTEzgGOAMw0EuSJEntoMlAn5nTG/u5ORFxMfClGtQlSZIkqQot+lJsROwEbAkMoHT1/pHMvGdRf2Y+AXy5phVKkiRJalJVgT4i1gT+C9gCeKP86AecFhFTgb0zc0abVSlJkiSpUStUOe4KSlflh2dm/8z8Qmb2B7YF+gOXt1WBkiRJkppWbaD/KnB8Zv65sjEzpwAnAtvXujBJkiRJS1dtoP8HMK+JvnnA7NqUI0mSJKklqg30Z1CaLz+wsjEiBgHjgJ/XuC5JkiRJVWjuTrE3NGhaDfh/EfFX/vml2M2BWcBISvPsJUmSJLWj5la56dvg9QvlB0Av4ENg0Zz61WtclyRJkqQqNHdjKb/oKkmSJC3nqp1DL0mSJGk5tNRAHxFfioirIuL5iPig/Hg+Iq6MiC+1R5GSJEmSGtdsoI+I44CpwL7A/1H64usV5Z/3Bf5SHiNJkiSpAzS3ys3uwNnAOcAZmflug/6ewFjgrIh4OjNva9NKJUmSJC2huVVufgRcm5knNtaZme8BJ0XEAOBYwEAvSZIktbPmptxsBkyoYh8TKK1HL0mSJKmdNRfoVwAWVLGPBUvZjyRJkqQ20lwQfwr4ehX7+DqlL8lKkiRJamfNBfrxwBER8e2IiMYGRMRhwPeAy9qiOEmSJEnNa+5Osb+KiKHA5cCxEXELML3cvRawG7AecHlmXtfmlUqSJElaQnOr3JCZ342Iu4AfAEcAXctdHwEPASdm5s1tW6IkSZKkpjQb6AEy8w/AHyKiE7B6uXl2Zn7SppVJkiRJWqqlBvpFygH+H21YiyRJkqQWcrlJSZIkqcAM9JIkSVKBtXugj4iNIuKPETE3ImZGxGnl+fnNbbNFRFwTEdPK2z0XEadGRLdmttksIj6JiNmN9GUjj4dr8f4kSZKk9lT1HPpaiIg+wL3A08CewLrAeZQ+WJzSzKajymPPBl4AvgCcXn7et5HjBHAJMIum3+N5wI0Vr99rwVuRJEmSlgvtGuiBw4HuwD6Z+S5wT0T0AsZFxDnltsaclZmVV9rvi4gPgcsjYq3MnN5g/LeANYCrgTFN7PPlzPSqvCRJkgqtvafc7ALc1SC4T6AU8rdraqMGYX6Rx8vPa1Y2RkRPSlfyjwU+XqZqJUmSpOVcewf6DYBnKxsy8+/A3HJfS2wNLARebND+E+CZ8vr5zRkXEQsiYnZEXB0Rq7bw+JIkSVKHa+8pN32Atxtpn1Puq0pE9Kc05/7XmflGRfv6lO5oO3Qpu7gWuIXSHPs64MfAFyNiS2+YJUmSpCJp70C/zCKiC3AD8D5wTIPui4BfZebfmttHZh5c8fL+iHgGuB3YHVjiyn5EjKE8F/9zn/tcq2uXJEmSaq29p9zMAXo30t6n3Nes8uo11wEbA7tm5pyKvl2AYcD5EbFKRKwCdCtvtkpEdG1m13dS+oCweWOdmXlFZtZlZl3fvn2XVqYkSZLUbtr7Cv2zNJgrHxGDgR40mFvfhAspLXe5Y2Y2HL8+sDKlZS0bmkNpWs3PGttpZmbpswJZRQ2SJEnScqO9A/0dwHER0TMzF637PgqYB0xubsOIGAscCRyQmQ82MuRG4IkGbQcDe1P6EPBSM/vemdKHgceqeA+SJEnScqO9A/144ChgUkScDawDjAPOr1zKMiKmAZMz89Dy628AZwC/AmZExFYV+3wxM2dl5qvAq5UHi4gRwPzMvK+ibQylL8LeC8ymNM3mFOBR4LYavldJkiSpzbVroM/MORGxA6W7uN5CacWbCyiF+oZ1dap4vVP5+eDyo9K/Uwr61XoR+DdKd5jtBbxOaV7+j13hRpIkSUUTmU4bb4m6urqcOnVqR5chSZKkT7mIeCwz65Y2rr1XuZEkSZJUQwZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSowA70kSZJUYAZ6SZIkqcAM9JIkSVKBGeglSZKkAjPQS5IkSQVmoJckSZIKzEAvSZIkFZiBXpIkSSqwdg/0EbFRRPwxIuZGxMyIOC0iOi1lmy0i4pqImFbe7rmIODUiujWzzWYR8UlEzG6kr2tEnBcRb0TEBxFxW0QMWfZ3J0mSJLWvFdvzYBHRB7gXeBrYE1gXOI/SB4tTmtl0VHns2cALwBeA08vP+zZynAAuAWbR+Hv8BbAfcEx5zDjgnojYNDM/bMVbkyRJkjpEuwZ64HCgO7BPZr5LKUT3AsZFxDnltsaclZmVV9rvi4gPgcsjYq3MnN5g/LeANYCrgTGVHRExCDgUOCQzryu3/S/wUnm7q5btLUqSJEntp72n3OwC3NUguE+gFPK3a2qjBmF+kcfLz2tWNkZET0pX8o8FPm5ku53Kz5Mq9j8DeLBcnyRJklQY7R3oNwCerWzIzL8Dc8t9LbE1sBB4sUH7T4BnMvMPzdTwama+36D9mVbUIEmSJHWo9p5y0wd4u5H2OeW+qkREf0pz7n+dmW9UtK8PHAEMbesaJEmSpOVB4ZatjIguwA3A+5S+1FrpIuBXmfm3Gh9zTERMjYips2bNquWuJUmSpGXS3oF+DtC7kfY+5b5mlVevuQ7YGNg1M+dU9O0CDAPOj4hVImIVoFt5s1Uiomtra8jMKzKzLjPr+vbtu7QyJUmSpHbT3lNunqXBPPWIGAz0oMHc+iZcSGm5yx0zs+H49YGVKS1r2dAc4MfAz8rHGRwRK2XmBxVjlpjfL0mSJC3v2jvQ3wEcFxE9M/O9ctsoYB4wubkNI2IscCRwQGY+2MiQG4EnGrQdDOxN6UPAS+W2u8vPewO/Ke97TWBb4HsteTOSJElSR2vvQD8eOAqYFBFnA+tQuqnT+ZVLWUbENGByZh5afv0N4AzgV8CMiNiqYp8vZuaszHwVeLXyYBExApifmfctasvMVyPil8CF5Sk8i24sNZ1ywJckSZKKol0DfWbOiYgdKN3F9RZKq81cQClQN6yrU8XrRWvHH1x+VPp3SkG/JY4CPgDOpzTdZzJwoHeJlSRJUtFEZnZ0DYVSV1eXU6dO7egyJEmS9CkXEY9lZt3SxhVu2UpJkiRJ/2SglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgBnpJkiSpwAz0kiRJUoEZ6CVJkqQCM9BLkiRJBWaglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgBnpJkiSpwAz0kiRJUoEZ6CVJkqQCM9BLkiRJBWaglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgBnpJkiSpwAz0kiRJUoEZ6CVJkqQCM9BLkiRJBWaglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgBnpJkiSpwAz0kiRJUoEZ6CVJkqQCM9BLkiRJBWaglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgBnpJkiSpwAz0kiRJUoEZ6CVJkqQCM9BLkiRJBWaglyRJkgrMQC9JkiQVmIFekiRJKjADvSRJklRgkZkdXUOhRMQsYHoHHX51YHYHHVvtw3P82eB5/mzwPH/6eY4/GzryPK+VmX2XNshAXyARMTUz6zq6DrUdz/Fng+f5s8Hz/OnnOf5sKMJ5dsqNJEmSVGAGekmSJKnADPTFckVHF6A25zn+bPA8fzZ4nj/9PMefDcv9eXYOvSRJklRgXqGXJEmSCsxAvxyIiI0i4o8RMTciZkbEaRHRqYrtekfENRExJyLeiYjfRsRq7VGzWqY15zgitiif32nl7Z6LiFMjolt71a2Wae3vcsX2K0TE1IjIiPh6W9aq1luW8xwR+0TEXyJiXkS8GRF3RsRKbV2zWm4Z/t9cFxF3R8Rb5ce9ETG0PWpWy0TE5yPi8oj434j4JCLuq3K75S5/rdiRBxdERB/gXuBpYE9gXeA8Sh+2TlnK5jcA6wGHAQuBs4E/ANu2Vb1quWU4x6PKY88GXgC+AJxeft63DUtWKyzj7/IihwGD2qRA1cSynOeIOAy4BDgHOA7oA3wV/1+83GnteY6IweXt/gr8a7n5OOCeiNg0MzvqPjZq3MbArsDDQOcWbLf85a/M9NGBD2AsMAfoVdF2PDC3sq2R7bYGEvhKRduW5baRHf2+fNTkHK/eSNuY8jleq6Pfl4/anOeKsX2AWcCh5XP89Y5+Tz5qd54p3ZjmPeDbHf0efLTpeT4c+AToXdHWp9z23Y5+Xz6WOF8rVPx8I3BfFdssl/nLKTcdbxfgrsx8t6JtAtAd2G4p2/0jM+9f1JCZjwIvlfu0/GjVOc7Mxu5K93j5ec3alacaae3v8iKnA1OAP7ZBbaqd1p7nA8rP17ZVYaqp1p7nzsAC4IOKtvfLbVHrIrVsMnNhKzZbLvOXgb7jbQA8W9mQmX+ndBVgg5ZsV/bMUrZT+2vtOW7M1pT+vPdibUpTDbX6PEfEF4BDgGPbrDrVSmvP81DgOeDQiHg1IuZHxCMRsU3blapl0NrzfFN5zHkR0S8i+gEXULra//s2qlXta7nMXwb6jtcHeLuR9jnlvlpvp/ZXk3MVEf0pzd38dWa+UaPaVDvLcp4vBi7JzGk1r0q11trz3B9Yn9Lv8AnA7pSu4t4ZEWvUukgts1ad58ycCWxP6XtO/yg/9gG+lpmz2qBOtb/lMn8Z6KUCiIgulL6E8z5wTAeXoxqKiNGUgt7POroWtakAVgYOzczfZuadwF6U5lYf2aGVqWYiYgClK/GPUZp+sUv559si4nMdWZs+3Qz0HW8O0LuR9j7lvlpvp/a3TOcqIgK4jvK38TPT87t8avF5jojOwLmUVkhYISJWAXqVu1eKiJ5tUaiWybL8NzuB+xY1lOdnPwZsVMP6VButPc/HUZpHv19m3ln+4LYvpQ9uTqn7dFgu85eBvuM9S4M5V+Vlr3rQ+BytJrcra2pulzpOa8/xIhdSWjZtz8z03C6/WnOeV6K0TOX5lP5HMAd4stw3gX9+CVrLj9b+Pj9D6Sp9wy9GBqXvxWj50trzvAHwVGbOX9SQmR8DT1Fa+lLFt1zmLwN9x7sD+FqDK3GjgHnA5KVs1z8ihi9qiIg6YJ1yn5YfrT3HRMRYSn+O/1ZmPth2JaoGWnOe36c037bycWC57yTgm21TqpZBa3+fby0/b7+oISJ6A1/mnx/itPxo7XmeDmxSniYJQER0BTYBXm6DOtX+lsv8FeX1M9VByjeveBr4P0p/dl+H0tW6CzPzlIpx04DJmXloRdtdwL9Q+jPeohsbvJGZ3lhqOdLacxwR3wB+C/wKuLzBbl/0C1bLl2X5XW6wnyGUlj/bPTNvbWyMOs4y/jf7D/z/9u491LKyjOP495f+MWNRYDhKqDhhVkIXq0nsTl6SEcILXVCpmS4g3UEKiWKGGkioFPpDiOgyFWnBeMkxphkLIittUCGdCGlsKgq1yBxrnMTm6Y93bdru2TPn6tl7nfl+4HDWevd7W2dzDs959/Ou1e52cxXwd9p9zU8HTjOVbros4O/2q2kPKdoOXEf7BObDwDnAa6rKf96mSJJjaA+WAriSlvK4oTv/UVXt60v85dPpJqyqHk1yNu3pgbfSdk5fC2wcqXo0MPrI6Xd1db9B+7RlK/CxZ3K+mrsFvMfndd/XdV/D1tMCfU2JBf4uqycW+D5fTtszcQ0tdeMXwFsN5qfPfN/nqro7yfm0oPA7XfF9wLkG81NpFQffTnRwvpr2qUov4i9X6CVJkqQeM4dekiRJ6jEDekmSJKnHDOglSZKkHjOglyRJknrMgF6SJEnqMQN6SZIkqccM6CWpZ5LULL7esgjjPJRk0xzbrOjG/8BCx5/DmHcm+e4c25zfzfPUZ2pekrRUfLCUJPXPWUPHK4GfApuA24bKf7sI46wFHpljm//Q5rd7EcafrfcD+5dwPEmaKgb0ktQzVXXn4DjJc7rD3cPlh5JkRVXNKvitqnvmMbcCZpzHYqqqXUs5niRNG1NuJGmZSnJFl1byqiQ/T/IE8NE0X05yf5J/J/lzks1Jjhtp/7SUmyQ3JLkjydoku5L8K8nPkrx4qM5BKTeDlJgk703yYJK9SW5NcsLIeC9MsiPJE0l2J7k0ydYk22a4zoNSbpKcl2Rnkv3ddXwlycoxzU9Ksi3JviR7krxvdj9dSZoeBvSStPx9H9hCS6HZTvvbfywtTWctcCVwOrAjSWbo69Su3UbgcuAk4HuzmMObaKkxnwA+REvLuW7wYpJnAVuB1cA64FPAVcArZ9H30yQ5g5Z+9BfgYuDzwHrg+jHVNwN3ARcBPwG+nuScuY4pSZNkyo0kLX9fqqqvjpStHxwkOQq4G/g9sAb49WH6OhY4s6r+2LVdAVyf5JSq2nOYds8GLqiqx7t2JwKbkhxdVU/RAuqXAq+oqt90de7p5nT/rK+02QA8AFxcVQe6vh4HNic5o6ruHap7c1Vt6I5/nORFwGeA2+c4piRNjCv0krT83TZakOTtXarKY8BTtMAZ4LQZ+npgEMx3BptvT5yh3a8GwfxQu6OAQdrNGmDPIJgHqKo/APfN0O84rwW2DIL5zg+AAt4wUvemMedr5jGmJE2MAb0kLX8PD58keT0tcN1NS5s5i5YSA7Bihr7+OXL+5CK1OwH425h248oOqUsZOp6Ra+42Au+lfcIwbPQuPo8AxyR53lzGlaRJMuVGkpa/Gjm/BPhTVV02KBje2DohDwFvHlN+XPfarFRVJXkYWDVc3qUGPRf4x0iTVcCukfN9VfXYbMeUpElzhV6Sjjwr+f8K+cBl4youoZ3AKUlePihIshp42Tz6ugu4ZGSD7zuAAHeM1L1ozPnOeYwpSRPjCr0kHXl2AFck+SKwjZZu8+7JTombgN8BNyb5NC2vfyNtdf7AYdqN8zlaUL4lyddod865GrhlZEMswIVJHgV+CbwTeCPwtvlehCRNgiv0knSEqaobgc/SVuV/CJwJXDjhOR0ALgD2AN8GrgGupeX5751NF0N93dv1dTJwM+2uN98CLh3Tbh3wuq7eucAHq2r7/K5CkiYj7aF+kiRNlyTPBx4Erq6qLxym3i7g9qr6+JJNTpKmiCk3kqSpkOQjwH7aLTSPBz7ZvbT5EPVfQFtVfwktpUaSjkgG9JKkafEkLYg/GfgvbXPr2VX110PUfw/tibLfBG5YkhlK0hQy5UaSJEnqMTfFSpIkST1mQC9JkiT1mAG9JEmS1GMG9JIkSVKPGdBLkiRJPWZAL0mSJPXY/wBLFLlS/usl5AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x504 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# target vs iteration\n",
    "\n",
    "ax = (results.sort_values(by='TrainingStartTime', ascending=True)\n",
    " .reset_index()[['FinalObjectiveValue', 'cumulative']].plot())\n",
    "ax.set_xlabel('Training job')\n",
    "ax.set_ylabel('Objective Value');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This allows us to display epoch-by-epoch performance of all iterations and highlight the best iteration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze continous hyperparameters\n",
    "\n",
    "<font color=\"brown\"> **Make sure to use variables that were actually tuned in your specific tuning job for `x` and `y` below. Also check if `logx` and `logy` make sense for the parameters you're analyzing. Depending on the number of points, a higher alpha may be a good idea as well.**</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwcAAAHLCAYAAACd7eSpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3XuYXWV99vHvHRKEAAnRAAoCQYIGaOXQeMJDEFAEpSgIWEWM2qqvth7wAOVV30hRBIvaihZEJFZFqUCNyBkRKkgswZYgGFsR0AjIwUCAhFPye/9YKzgZ9kz2JJPZSfh+rmtfM/tZz1rrtzd7wrr3ep61UlVIkiRJ0qheFyBJkiRpzWA4kCRJkgQYDiRJkiS1DAeSJEmSAMOBJEmSpJbhQJIkSRJgOJAkAJLsmaSSTO91LUPV1j2z13VozZXkiiS39mubmcTrmUtajuFAkrTGakPbjCSb9roWSXoqGN3rAiRJq2xDYEmvi1hN9gT+HzATuK+nlazdXg2k10VIWvN55kCS1hBJNlmZ9arq4ap6bLjrGW4r+/q06qrq0ap6pNd1SFrzGQ4kaRBp/J8k1yVZlOTBJD9O8soOfd+b5JIkv0/yaJI7knwryaQOfasd8713kquSPAic1y6b0S5/XpLPJJmf5JEk1yfZf6BtDbD9lyS5MslDSe5N8rUkG3fYxrQk1yRZnOTOJP+UZOd2OzOG+J5NWrZeksPa924x8KV2+ZQkX0lyY5IH2vf1uiR/3W87M2nOGgDc0m5zuXqSjE9yQpJft+/R3Um+k+Q5XdR5Qru953dYNr59L77fp+217Xt5T7vst0nOTfLcLt+XqUn+vV3/kSS/SvJ/k4zu1++KJLcmeU6SWUnuT7KwXfc5/fqOSvLBJHPb93Jhu93Tk4zpv80u63x+u697kzyc5KYkH0uyXr9+M9v3b3ySf0lyV9v/6iQv6mZfktY8DiuSpMF9E/gr4GzgDOBpwFuAS5McVFU/6NP3I8Bs4J+BPwJ/Bvw1sFeSP6+qe/tteypwMHAa8I0O+/4G8Bjwj8D6wAeB7yd5blXd2kXtuwI/bOs+k2aIzjuBpcC7lnVK8jLgEmAB8Fma4TuHAi/tYh+DeT3wfuBfgFOAhW37nsAr2tpuATYCDgFOS7JZVR3f9jsVGAe8AfgQcE/bPretezzwU2Ab4OvAjcCzgPcCP0sytapuG6S+bwAfA46g+W/X16HABm0fkkwDfgD8Ajie5j3aEtgHmAz8z2BvRJLXAucCvwZOovl8vAQ4lua/0yH9VtkIuAL4GfD3wA7t63pxkt2q6s623/9tt3EezXu8BNgO+Euaz+qQziglmQpc2a73ZeBO4ADgBGAXms9+fxcDd7d1PAM4Ejg/yXZV9cBQ9i9pDVBVPnz48PGUf9AcsBYwvU/bG9q2d/XrOxqYQ3Ngmz7tG3XY7t7tNj7Wr73axz4d1pnRLvthv+2/oG0/vsO2ZnZoWwq8qF/7+TQHfhv3aftP4GHgOX3axgBXt9uZMcT3clK73mPAjh2Wd3qfRtEcDN8PjOnwXkzqsM4/AYuBXfq1b0sTRGZ2Ueu1wO3Aev3af0ITRtZvn3++rWPzlfhsbUBzkP0fwOh+yz7UbnfPPm1XtG1f7Nd32efxlD5tPwdu6qKGK4Bb+7XNbA4Dlmu7GngceH6ftgD/1u577/7rA1/pt41D2vZ3D/W98uHDR+8fDiuSpIEdDjxA8239xGUPYFOab2on0XyjC0BVPQRPDPUY3/a9nuaAt9Mwi+ur6rJB9v9PVfXEpSar6lrgwb77XIFrqupn/doupwk3k9pat6AJHbOq6jd99vUYzcH3qji/qn7Zv3HZ+9Tuf4MkzwCeTnP2YhwwZUUbThKab7H/A/h9v/8+D9GcwXl1FzV+g+Zsw6v6bHs7mrMm36mqR9vm+9ufB/cfBtSFVwFb0JzB2bRfrRe0fTrV+tm+T6rq34Ff0ZyRWeZ+YKv27M8qSbI5sAfwg6qa22e/BXy6ffqGDqt+od/zy9uf3X5OJa1BDAeSNLAdgU2AP9AMm+j7mNH22WJZ5yR7JbmC5uD0vj59xwMTOmx/0KEowG86tN1LM3SjGwOtT59tbNf+/FWHvp3ahqLj60uycZJ/TPJbmm/+76F5n5YdgHZ6r/rbjOY1vJon/7e5mz8dkK/Id4BHaYYWLXMEzbfl/9qn7WTgv4CvAH9MckGS9yfZrIt97Nj+/HqHOue1y/rXel/9aehQX78EtkiyUfv8GJqzPj9JM9fl20nenGT9Lurqb9ln4cYB9rsU6DSXY7nPWf1p+Fy3n1NJaxDnHEjSwEJzAPfmQfr8AiDJC2i++f41cDTNkKPFNMMrvkvnL2MWrWD/A12etNtLUg52edORuKzlQK/vTOB1wFdpvvm/l6bW/WmG2XTzxdWy+i+jGQ+/Uqrq3iQXAK9Pskk1Y+TfCvyyPVPTt98LgJfTBI9X0Hxj/qkk+1fVNV3U+lHgvwfoc/tK1n9Nku2BfYFXto83Ax9P8rKq+uPKbHeINazq51TSGsRwIEkD+1/gucDsqnpwBX3fDKwH7FdVtyxrbL/h7eab8F65tf35vA7LOrWtkjQ3M3sd8M2qek+/Zft0WGWgO/jeTXN2ZtwKhmZ14xs0Q3UOSfIrYHuagLd8Ic1B8BXtg/YqR9cBHwdeO8j2/7f9+dAQat00yTM7nD3YEbir79Cs9rN5TvsgyXtpJhO/E/hcl/uDJtAC7Nxh2RSa0NbpbJSkdYjDiiRpYP9K8+/k8Z0WtuP1l1n27Wn/b0uPYQ3+t7Y9+JwDHNj3MpntZTA/sBp22fF9SvIsmis79bcslD29b2NVLQW+DbwwyRs77agdQ9+N82mGNh3RPpYC3+q3rYkd1ptHc3bo6R2W9XUxcBdwdJIn9U2yYTrfA+Lofv3eQBPY+l5etVNdP29/rqiu5VTVXTRXfzogyZ/12UdorpgE8O9D2aaktY9nDiRpAFV1dpIzgL9NsjvN1YPuAZ5NcxnKyfxpDPa/0wyJuSDJV2nGsb8KeD5/ugTnmuojwKXAT5N8hWaS66E0l0+Fgb+9H7KqeiDJJcDhae59cC3N1YXeTfPNdf9x6rPbnyck+TbN+PpfVNUvaC7j+VLg35L8W9v30XZ7+9N8qz+9i5oeS/Id4G+BvwAuq6rf9+t2WpJn0wwdu43mrtSH0cxJ+VcGUVUPJTmC5qD+V0m+TjP8bFOab+QPopnoe0Wf1e4BDkqyZdu+7FKmf+BP810AfplkNs0lT2+nmVz9rvZ9+O6KXnsHH6C5lOlPkiy7lOnraIYtnVlVP1qJbUpaixgOJGkQVfWOJD+mOeD6e5oD5jtpvp39+z79rk5yMPAJ4B9ovlG+DJhGM65+jVVVVyZ5DfAZmjMd9wFn0cwNmE3zWobT4TRX4jkAeBvNsJv/S3Pp0zP61XZ1kqOA99DcD2I08CmagHB/kpcCH6YJMwfSXIZzPnAV8LUh1PQN4O+Ajel8sP9NmqDxNprJ0AuBm4A3VtU5K9p4VV3czlk4mub1b0ZzX4mbaS6TOrffKg8Be9HMa/gszZmWi4APV9UdffqdRBOE3k8z8f0umv9mx1fV9Suqq0Odc5LsQfMev5fmfgu/AY5q9yVpHZc+V8mTJOkJbdg5G/irqlqZb6G1EtorXk2qqkk9LkXSU9AaOw5WkjQy0tigX9sYmjvdPs7yw10kSeswhxVJkp4G3NaO6f8Vzbj/w2jmS5yw7Io5SZ7Zxbbur6rhHoYkSRohhgNJ0mM0V+w5kGZCa2hCwvuq6it9+t3RYd3+3g7MHO4CJUkjwzkHkqSuDHAfgv5u7DdhVpK0FjEcSJIkSQIcVtRzEydOrEmTJvW6DEmSJK3DrrvuunuqarMV9TMc9NikSZOYM2dOr8uQJEnSOizJbd3081KmkiRJkgDDgSRJkqSW4WAYJXltkuuT/HeSnybZsdc1SZIkSd0yHAyv04A3VdWuwLeAY3tcjyRJktS1noSDJKOTHJ3kf5M8kmR+ki+shv1MTnJqkrlJliS5YpC+OyX5UZJFSW5PcmyS9Ya4y6XAuPb38XR3wyBJkiRpjdCrqxXNBPYCPgXMA7YGdloN+9kZ2B+YDYwZqFOSCcBlwE00dwjdHjiJJjx9fAj7Oxz4YZKHgUXAS1eubEmSJGnkjXg4SPIa4DBgl6q6qct1dgPurqr5HZbtD1xcVUs6rHpeVc1q+50NTBxgF+8BNgQOqqqFwKVJxgEzkpzYtpHk58A2Hda/GHgbcBSwV1XdkORvaELQ67p5jZIkSVKv9WJY0TuAy7sNBq0ZNAfsyx3cJzkCOI/mLMSTVNXSLre/H03AWNin7bs0gWFan+3tXlUTOzzeAuwKPL2qbmi7fxt4ZZf7lyRJknquF+HgRcD/JDk5ycJ2jP+5SbYcZJ3pwGPARe03+iQ5EPg6cExVXbqKNU2hGd70hKr6Lc3QoCldbmM+MDnJVu3z19AMU5IkSZLWCr0IB8+kOdjfFXgT8HbgL4B/T5JOK1TVAmBfYALwg3Yo0VnASVV1wjDUNAG4r0P7gnbZClXVncBHgEuSXN/+/s6B+ic5IMlX77///pUoV5IkSRp+vQgHaR8HVtUFVXUW8FbghQwwPAigqu4A9gGeC5wPfLOqjhqBertWVWdU1c5VtUtV7VFVcwfpe15VvWv8+PEjWeJKuemmm9h7770ZO3YsW265JZ/85CdZsqTTFI8/ufbaa3n729/O5MmTGTt2LM973vP41Kc+xcMPP7xcvxkzZpDkSY+LLrpodb4kSZIkddCLqxUtAH5TVff2absKeJTmikU/GmTdDYH1aS4ZunGSUUOYV7CimjodpU9olz1lLViwgH322YeddtqJWbNmcfPNN/PhD3+YpUuXctxxxw243llnncXNN9/MUUcdxQ477MDcuXP5xCc+wdy5cznnnHOW6zt+/PgnhYEdd/T+cZIkSSOtF+Hgl8AGHdpDc9DfUZJJwCU0lyX9LHAh8GXg/wxDTfPoN7cgydbAWPrNRXiqOeWUU1i8eDHnnnsu48aN41WvehULFy5kxowZfOxjH2PcuHEd1zv66KOZOPFP88f33HNPNthgA9797ndz2223se222z6xbPTo0bz4xS9e7a9FkiRJg+vFsKIfAn/e78pDr6C5D8H1nVZIsgVwKXAzcEhVXUVzP4K3J/nMMNR0IbBvkk36tB0GLAauHIbtr7UuvPBC9t133+VCwJve9CYWL17MlVcO/Nb0DQbL7LbbbgDcfvvtw1+oJEmSVlkvwsFXgXuB89pJuW8Gvglc1h70d3I6cD9wQFUtBqiqy2kO4D+aZN9OKyUZm+SNSd4IbAVstux5krF9up4CPAKcm2SfJO+iuXzq5/td3vQpZ968eUyZsvwFm7bZZhvGjh3LvHlDO6lyzTXXMGrUKLbffvvl2u+77z4mTpzImDFj2G233Tj33HNXuW5JkiQN3YgPK6qqhUn2Av6Z5l4CjwKzgA8NstqRwB/7H6hX1awk04BrBlhvc+B7/dqWPd8OuLXdzoIkewMn09w34T7gCzQB4SltwYIFbLrppk9qnzBhAgsWdD8d48477+S4447jrW99K5tvvvkT7ZMnT+bEE09kt91244EHHuDUU0/l4IMP5pxzzuGggw4altcgSZKk7vRizgFV9Wtg/yH0/59Blv10kGW30sxl6GYfNzHI1ZK08h599FEOPfRQNt54Y77whS8st+zwww9f7vkBBxzAHnvswbHHHms4kCRJGmE9CQdac/3v//6Rn171W34//wE22nh9Nt54PPfe++QzBAsWLGDChBXfAqKqOOKII7jxxhu5+uqrV7hOEg466CCOOuoolixZwnrrrbfSr0WSJElD04s5B1pDXXnFbXxz5vX88d6H2XyLjRgzZhTjxm3FxRfNZvHix57o97vf/Y5FixY9aS5CJx/84AeZNWsWs2bN6qo/8MS9DiRJkjSyDAcC4O67HuLHl9/C1tuMZ9MJG7DeeqPYcMMxvGDqK7jhF9dwxY9/9UTfs846iw033JBp06YNus3jjz+ek08+mW9961u87GUv66qOquKcc85hl1128ayBJEnSCHNYkQC48ca7GTN6PUaPXj4v7jntEC697Du8/wNv5ytjPsMtt9zCjBkzOPLII5e7vOnkyZOZNm0ap59+OgBnnnkmxxxzDNOnT2errbZi9uzZT/Tdfvvt2WyzzQCYNm0aBx98MFOmTOGhhx7itNNO42c/+xnf//73R+BVS5IkqS/DgQBYeP8jrP+0J39Tv9FG4/joR07l9K9/mr/8y79k00035UMf+hAzZsxYrt/jjz/OkiVLnnh+ySWXADBz5kxmzpy5XN8zzjiD6dOnA02o+OIXv8gdd9zBqFGj2H333Tn//PPZb7/9hvX1SZIkacVSVb2u4Slt6tSpNWfOnF6Xwc9m/55LLvo1z956/JOWPfTgoyyt4v0ffFEPKpMkSdKqSnJdVU1dUT/nHAiAHXeayHrrjWLRoseWa1+6tLjrrkW87OVb96gySZIkjRTDgQAYN+5pvPHQnViw4GHmz1/I/fc9zF1/eIjf/fY+/uIFz2LX3Z7V6xIlSZK0mjnnQE947vOewfv+7gX84oY/MP93C9lo4/XZZdct2Gab8V5aVJIk6SnAcKDlTJiwAS9/xba9LkOSJEk94LAiSZIkSYDhQJIkSVLLcCBJkiQJMBxIkiRJahkOJEmSJAGGA0mSJEktw4EkSZIkwHAgSZIkqWU4kCRJkgQYDiRJkiS1DAeSJEmSAMOBJEmSpJbhQJIkSRJgOJAkSZLUMhxIkiRJAgwHkiRJklqGA0mSJEmA4UCSJElSy3AgSZIkCTAcSJIkSWoZDiRJkiQBhgNJkiRJLcOBJEmSJMBwIEmSJKllOJAkSZIEGA56JskBSb56//3397oUSZIkCTAc9ExVnVdV7xo/fnyvS5EkSZIAw4EkSZKkluFAkiRJEmA4kCRJktQyHEiSJEkCDAeSJEmSWoYDSZIkSYDhQJIkSVLLcCBJkiQJMBxIkiRJahkOJEmSJAGGA0mSJEktw4EkSZIkwHAgSZIkqWU4kCRJkgQYDiRJkiS1DAeSJEmSAMOBJEmSpJbhQJIkSRJgOJAkSZLUMhxIkiRJAgwHkiRJklqGA0mSJEmA4UCSJElSy3AgSZIkCTAcSJIkSWoZDiRJkiQBhgNJkiRJLcOBJEmSJMBwIEmSJKllOJAkSZIEGA4kSZIktQwHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJkiRJgOFAkiRJUstwIEmSJAkwHEiSJElqGQ4kSZIkAYYDSZIkSS3DgSRJkiTAcDCskrw2yfVJ/jvJT5Ps2OuaJEmSpG4ZDobXacCbqmpX4FvAsT2uR5IkSepaT8NBkq2SPJikkmy8GrY/OcmpSeYmWZLkikH67pTkR0kWJbk9ybFJ1hviLpcC49rfxwN3rGTpkiRJ0ogb3eP9fw54ENhoNW1/Z2B/YDYwZqBOSSYAlwE3AQcC2wMn0YSnjw9hf4cDP0zyMLAIeOnKlS1JkiSNvJ6dOUjyCuA1wD920Xe3JM8eYNn+g3zDf15VbV1VhwA3DrKL9wAbAgdV1aVVdQrwKeDIJMvOBJDk50nu6fD4dpLRwFHAXlW1dfu6Zq7otUmSJElrip6Eg/Zg/ks0Y/Lv6WKVGcClSSb2284RwHnAXp1WqqqlXZa0H3BxVS3s0/ZdmsAwrc/2dq+qiR0ebwF2BZ5eVTe03b8NvLLL/UuSJEk916szB+8BngZ8ucv+04HHgIuWfZOf5EDg68AxVXXpKtYzBZjXt6GqfkszNGhKl9uYD0xOslX7/DU0w5QkSZKktcKIzzlI8gzgH4DDq+qxJCtcp6oWJNkXuAr4QZITgbOAk6rqhGEoawJwX4f2Be2ybmq8M8lHgEuSPA48BLxzoP5JDgAOmDx58kqUK0mSJA2/Xpw5+DQwu6ouGMpKVXUHsA/wXOB84JtVddRqqG+lVdUZVbVzVe1SVXtU1dxB+p5XVe8aP378SJYoSZIkDWhEzxwk2Rl4B/CKJJu2zWPbn+OTLKmqxYNsYkNgfZpLhm6cZNQQ5hUMZgHNpUf7m9AukyRJktZ5I33mYAeaS4peQ3PQvYA/zTuYTzNJuaMkk4BLaC5LOg14Hd3PWViRefSbW5Bka5rgMq/jGpIkSdI6ZqTnHFzFk6/g8xqaS4DuD/ym00pJtgAuBW4GDqmqxe2E5AuSLKiqY1axrguBjybZpKoeaNsOAxYDV67itiVJkqS1woiGg6q6B7iib1t7RgDgJ1X14ACrng7cDxywbNhRVV2e5DDg7CRXVtXF/VdKMpYmdABsBYxL8sb2+QVVtaj9/RTg/cC5SU4AnkNz+dTP97u8qSRJkrTO6vUdkrt1JPDH/gfqVTUryTSaYUqdbA58r1/bsufbAbe221mQZG/gZJr7JtwHfIEmIEiSJElPCamqXtfwlDZ16tSaM2dOr8uQJEnSOizJdVU1dUX9enUTNEmSJElrGMOBJEmSJMBwIEmSJKllOJAkSZIEGA4kSZIktQwHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJkiRJgOFAkiRJUstwIEmSJAkwHEiSJElqGQ4kSZIkAYYDSZIkSS3DgSRJkiTAcCBJkiSpZTiQJEmSBBgOJEmSJLUMB5IkSZIAw4EkSZKkluFAkiRJEmA4kCRJktQyHEiSJEkCDAeSJEmSWoYDSZIkSYDhQJIkSVLLcCBJkiQJMBxIkiRJahkOJEmSJAGGA0mSJEktw4EkSZIkwHAgSZIkqWU4kCRJkgQYDiRJkiS1DAeSJEmSAMOBJEmSpJbhQJIkSRJgOJAkSZLUMhxIkiRJAgwHkiRJklqGA0mSJEmA4UCSJElSy3AgSZIkCTAcSJIkSWoZDiRJkiQBhgNJkiRJLcOBJEmSJMBwIEmSJKllOJAkSZIEwOihdE7yRuAg4NnABv2XV9ULh6kuSZIkSSOs63CQZAbwSeB64Cbg0dVU01oryWuBzwAFLALeWVW/7G1VkiRJUneGcubgncBnq+qY1VXMOuA0YO+q+mWS9wLHAof0uCZJkiSpK0OZc7AJ8KNV3WGSNyb5aZJ7kzyc5FdJPp5k/VXddod9TU5yapK5SZYkuWKQvjsl+VGSRUluT3JskvWGuMulwLj29/HAHStZuiRJkjTihnLm4LvAa1j1gPAM4HLgc8B9wAuBGcAzgb9dxW33tzOwPzAbGDNQpyQTgMtohksdCGwPnEQTnj4+hP0dDvwwycM0w4peunJlS5IkSSNvKOHgR8AJSSYCl9Ic2C+nqi5Y0Uaq6tR+TT9OMg54X5K/q6rqv06S3YC7q2p+h2X7AxdX1ZIOuzuvqma1/c4GJg5Q1nuADYGDqmohcGlb04wkJ7ZtJPk5sE2H9S8G3gYcBexVVTck+RtgJvC6AfYpSZIkrVGGEg7Oan9OojkQ7q+AoQ7DWeZeYLBhRTOA5yZ5eVXds6wxyRHAGTRnNC59UkFVS7vc/340AWNhn7bvAicA04Dz2u3tPtAGkkwFnl5VN7RN3wa+2OX+JUmSpJ4bSjjYbjh33I7nfxqwO/B+4F86nTVoTQeuBC5KsldVLUxyIPB14JiqelIwGKIpNEOdnlBVv02yqF12XhfbmA9MTrJVVf2eJrDctIp1SZIkSSOm63BQVbcN874fogkHAP8KfHSQfS9Isi9wFfCDJCfSnMk4qapOGIZaJtBhmBSwoF22QlV1Z5KPAJckeZzm9b1zoP5JDgAOmDx58kqUK0mSJA2/od4EbTRwMPAy4OnAH4GfAOdW1eND3PcewFiaCcmfBE4G3jtQ56q6I8k+wNXA+cDXquqoIe5ztaqqM2iGOXXT9zzgvKlTp/7N6q1KkiRJ6s5QboK2OXAJ8HzgVuAPwEuA9wHXJ3l1Vd3d7faq6uftr1cluQf4RpKTqurmQVbbkGZuwlJg4ySjhjCvYDALaC492t+EdpkkSZK0zhvKfQ4+T3MZ0hdX1XOq6iVV9RzgRW3751ehjmVBYcB5DUkm0YST2TSThF8HfHkV9tnXPJq5BX33tzXNmY15w7QPSZIkaY02lHCwP3BUVf1n38aquhb4e+C1q1DHsvsB3NJpYZItaK5GdDNwSFVdRXM/grcn+cwq7HeZC4F9k2zSp+0wYDHNRGhJkiRpnTeUOQdPAx4YYNkDDH4p0ickuYjmhmM3AktogsGHgbMGGVJ0OnA/cEBVLQaoqsuTHAacneTKqrq4w77G0oQagK2AcUne2D6/oKoWtb+fQnPFpHOTnAA8h+byqZ/vd3lTSZIkaZ01lHAwGzgqyeVV9dCyxiQb0dz8a3aX27mW5tKkk4DHgd/QnHk4ZZB1jgT+2P9AvapmJZkGXDPAepsD3+vXtuz5djRzJ5ZdDWlvmknR59FcuegLNAFBkiRJekrIwLcW6Ncx2RX4Mc3Nzi6hmZC8ObAvEGDPqrp+NdW5zpo6dWrNmTOn12VIkiRpHZbkuqqauqJ+Xc85qKr/BnYAvgpsBryKJhycAuxgMJAkSZLWbkO6z0FV3QMcvZpqkSRJktRDQ7lakSRJkqR12KBnDpJcSzPHoCtV9cJVrkiSJElST6xoWNGNDCEcSJIkSVp7DRoOqmr6CNUhSZIkqceccyBJkiQJWPGcgxOBf66q+e3vg6qqjw1bZZIkSZJG1IrmHBwCfBuY3/4+mAIMB5IkSdJaakVzDrbr9LskSZKkdY9zDiRJkiQBQwgHSQ5O8s4+z7dL8tMk9yU5J8mmq6dESZIkSSNhKGcOPg6M6/P8S8BE4LPA7sCnh7EuSZIkSSNsRROS+3oOcANAkvHAq4E3VNX5SX5LExLeN/wlSpIkSRoJQ51zsOxuydOAJcBl7fP5wGbDVZQkSZKkkTeUcHA98JYkGwF/Dfy4qh5pl20D3DXcxUmSJEkaOUMZVnQMcB7wNuBB4FV9lr0e+Nkw1iVJkiRphHUdDqrqqiTbAM8Fbq6q+/os/jrw6+EuTpIkSdLIGcqZA6rqAeC6Du0XDFtFkiRJknpiSBOSk/x5kjOT/DrJQ+3Pbyd5/urr/du8AAAYSElEQVQqUJIkSdLI6PrMQZLXA/8G3AycTTMBeXPgQGBOkkOr6vurpUpJkiRJq91QhhWdAMwCDq2qZZc0JcnfA99rlxsOJEmSpLXUUIYVbQ18rW8wAGifn9YulyRJkrSWGko4mAPsPMCyPwN+vurlSJIkSeqVoQwrOhL4bpIxNMOHls05eAPNTdHelGTsss5VtWg4C5UkSZK0eg0lHPxn+/N44DN92tP+7H8TtPVWtihJkiRJI28o4eAdQK2wlyRJkqS10lDukDwTIMlOwF/QTED+elXdmWQy8If2JmmSJEmS1kJDuc/BRsAZwMHA4+26FwF30gwzug346GqoUZIkSdIIGMrVir4A7AHsA2zCn+YaAFwA7DeMdUmSJEkaYUOZc3AQ8IGq+nGS/pONbwO2Hb6yJEmSJI20oZw52BC4d4BlmwBLVr0cSZIkSb0ylHBwLXDEAMveCPx01cuRJEmS1CtDGVb0CeDSJJcB36O5rOn+ST5EEw5esRrqkyRJkjRCuj5zUFU/AfYGngacTDMh+VPAc4B9qura1VKhJEmSpBExlDMHVNXVwMuTbAhMAO6rqkWrpTJJkiRJI2pI4WCZqloMLB7mWiRJkiT10FAmJEuSJElahxkOJEmSJAGGA0mSJEktw4EkSZIkwHAgSZIkqWU4kCRJkgQYDiRJkiS1DAeSJEmSAMOBJEmSpJbhQJIkSRJgOJAkSZLUMhxIkiRJAgwHkiRJklqGA0mSJEmA4UCSJElSy3AgSZIkCTAcSJIkSWoZDiRJkiQBhgNJkiRJLcOBJEmSJMBwIEmSJKllOJAkSZIEGA4kSZIktQwHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJkiRJgOFAkiRJUstwIEmSJAkwHEiSJElqGQ4kSZIkAYYDSZIkSS3DgSRJkiTAcCBJkiSpZTiQJEmSBBgOJEmSJLUMB5IkSZIAw4EkSZKkluFAkiRJEmA4kCRJktQyHEiSJEkCDAfDKslrk1yf5L+T/DTJjr2uSZIkSeqW4WB4nQa8qap2Bb4FHNvjeiRJkqSujXg4SHJIkh8k+X2SB5Ncl+SvVtO+Jic5NcncJEuSXDFI352S/CjJoiS3Jzk2yXpD3OVSYFz7+3jgjpUsXZIkSRpxo3uwzyOBW4APAfcA+wNnJplYVV8a5n3t3G5/NjBmoE5JJgCXATcBBwLbAyfRhKePD2F/hwM/TPIwsAh46cqVLUmSJI28XoSDA6rqnj7PL0+yJU1o6BgOkuwG3F1V8zss2x+4uKqWdFj1vKqa1fY7G5g4QE3vATYEDqqqhcClScYBM5Kc2LaR5OfANh3Wvxh4G3AUsFdV3ZDkb4CZwOsG2KckSZK0RhnxYUX9gsEy/wVsOchqM2gO2Jc7uE9yBHAesNcA+1raZVn70QSMhX3avksTGKb12d7uVTWxw+MtwK7A06vqhrb7t4FXdrl/SZIkqefWlAnJLwH+Z5Dl04HHgIvab/RJciDwdeCYqrp0Ffc/BZjXt6GqfkszNGhKl9uYD0xOslX7/DU0w5QkSZKktUIvhhUtJ8newOuBdwzUp6oWJNkXuAr4QZITgbOAk6rqhGEoYwJwX4f2Be2yFaqqO5N8BLgkyePAQ8A7B+qf5ADggMmTJ69EuZIkSdLw6+mZgySTgDOBWVU1c7C+VXUHsA/wXOB84JtVddRqLnFIquqMqtq5qnapqj2qau4gfc+rqneNHz9+JEuUJEmSBtSzcJDk6cCFwG3AW7pcbUNgfZpLhm6cZLjqX0Bz6dH+JrTLJEmSpHVeT8JBkrHAD2kO9F9XVYu6WGcScAnNZUmn0VwF6MvDVNI8+s0tSLI1MJZ+cxEkSZKkddWIzzlIMhr4HrADsEdV3dXFOlsAlwI3A4dU1eJ2QvIFSRZU1TGrWNaFwEeTbFJVD7RthwGLgStXcduSJEnSWqEXE5K/QnNjsg8Az0jyjD7L/quqHumwzunA/TT3SFgMUFWXJzkMODvJlVV1cf+V2jMU+7dPtwLGJXlj+/yCPmcsTgHeD5yb5ATgOTSXT/18v8ubSpIkSeusVNXI7jC5Fdh2gMXbVdWtHdZ5LvDHTvdISLIHcE11eCHtUKRbutlXkp2Ak2kuq3of8DVgxgA3Vxs2U6dOrTlz5qzOXUiSJOkpLsl1VTV1Rf1G/MxBVU1aiXUGvAdCVf10kGW3AulyHzcxwM3UJEmSpKeCNeUmaJIkSZJ6zHAgSZIkCTAcSJIkSWoZDiRJkiQBhgNJkiRJLcOBJEmSJMBwIEmSJKllOJAkSZIEGA4kSZIktQwHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJkiRJgOFAkiRJUstwIEmSJAkwHEiSJElqGQ4kSZIkAYYDSZIkSS3DgSRJkiTAcCBJkiSpZTiQJEmSBBgOJEmSJLUMB5IkSZIAw4EkSZKkluFAkiRJEmA4kCRJktQyHEiSJEkCDAeSJEmSWoYDSZIkSYDhQJIkSVLLcCBJkiQJMBxIkiRJahkOJEmSJAGGA0mSJEktw4EkSZIkwHAgSZIkqWU4kCRJkgQYDiRJkiS1DAeSJEmSAMOBJEmSpJbhQJIkSRJgOJAkSZLUMhxIkiRJAgwHkiRJklqGA0mSJEmA4UCSJElSy3AgSZIkCTAcSJIkSWoZDiRJkiQBhgNJkiRJLcOBJEmSJMBwIEmSJKllOJAkSZIEGA4kSZIktQwHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJkiRJgOFAkiRJUstwIEmSJAkwHEiSJEnD6qabbmLvvfdm7NixbLnllnzyk59kyZIlg65z7bXX8va3v53JkyczduxYnve85/GpT32Khx9+eLl+M2bMIMmTHhdddNGw1D56WLYiSZIkiQULFrDPPvuw0047MWvWLG6++WY+/OEPs3TpUo477rgB1zvrrLO4+eabOeqoo9hhhx2YO3cun/jEJ5g7dy7nnHPOcn3Hjx//pDCw4447Dkv9hgNJkiRpmJxyyiksXryYc889l3HjxvGqV72KhQsXMmPGDD72sY8xbty4jusdffTRTJw48Ynne+65JxtssAHvfve7ue2229h2222fWDZ69Ghe/OIXr5b6HVYkSZIkDZMLL7yQfffdd7kQ8KY3vYnFixdz5ZVXDrhe32CwzG677QbA7bffPvyFDsBwIEmSJA2TefPmMWXKlOXattlmG8aOHcu8efOGtK1rrrmGUaNGsf322y/Xft999zFx4kTGjBnDbrvtxrnnnrvKdS9jOJAkSZKGyYIFC9h0002f1D5hwgQWLFjQ9XbuvPNOjjvuON761rey+eabP9E+efJkTjzxRL73ve9xzjnnsOWWW3LwwQcPW0BwzoEkSZK0Bnn00Uc59NBD2XjjjfnCF76w3LLDDz98uecHHHAAe+yxB8ceeywHHXTQKu/bMweSJEnSKli6tPjDnQ9y++8fYMKECdx///1P6rNgwQImTJiwwm1VFUcccQQ33ngjF1xwwQrXScJBBx3E3LlzV3i51G545kCSJElaSTfdeDcXX3QzDzzwKKHYaOyz+MlPfs7SpcWoUQHgd7/7HYsWLXrSXIROPvjBDzJr1iwuvfTSrvoDT9zrYDh45kCSJElaCb+86W6++50bGTNmFFtvPY5nbz2eXXd5OVdf/WPO+8ENT/Q766yz2HDDDZk2bdqg2zv++OM5+eST+da3vsXLXvayrmqoKs455xx22WUX1ltvvVV6PeCZA0mSJGnIli4tLrn4N2y+2Vg22mj9J9r33vtQfnzFdznyyHdAjuUPf5jPjBkzOPLII5e7vOnkyZOZNm0ap59+OgBnnnkmxxxzDNOnT2errbZi9uzZT/Tdfvvt2WyzzQCYNm0aBx98MFOmTOGhhx7itNNO42c/+xnf//73h+V1GQ4kSZKkIbr3nkXcf9/DbL3N+OXaN9poHB/5yKmc/vVPc9hhBzNhwqZ86EMfYsaMGcv1e/zxx5ebI3DJJZcAMHPmTGbOnLlc3zPOOIPp06cDTaj44he/yB133MGoUaPYfffdOf/889lvv/2G5XWlqoZlQ1o5U6dOrTlz5vS6DEmSJA3BH+58kK+e+nOe/ezOdzz+3W/v5w0HT+HPn7/FCFfWWZLrqmrqivo550CSJEkaoqc/Y0Oetv56PPLI409aVlVUwbOetUkPKls1hgNJkiRpiMaMWY9X7Lktt//+AR5/fOkT7UuXFvPnL2SnnScycbOxPaxw5TjnQJIkSVoJL3rxVjzyyBL+44rbWFpLgQDFn/35FrzugB16Xd5KMRxIkiRJKyEJ0/bclr+Y+izm/24hS5cWW2yxEc+YuPadMVjGcCBJkiStgo03Xp8pO07sdRnDwjkHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJkiRJgOFAkiRJUstwIEmSJAkwHEiSJElqGQ4kSZIkAYYDSZIkSS3DgSRJkiTAcCBJkiSpZTiQJEmSBBgOJEmSJLUMB5IkSZIASFX1uoantCR3A7f1ug4N2Xjg/l4XIWml+PertYGfUy0zXJ+FbatqsxV1MhxIKyHJV6vqXb2uQ9LQ+fertYGfUy0z0p8FhxVJK+e8XhcgaaX596u1gZ9TLTOinwXPHEiSJEkCPHMgSZIkqWU4kCRJkgQYDiRJkiS1Rve6AEmdJbkVWAQ82ja9uapu6l1Fkrrl36/WBn5O1YnhQFqz7V9Vt/a6CEkrxb9frQ38nGo5DivSWi3JIUl+kOT3SR5Mcl2Sv1pN+5qc5NQkc5MsSXLFIH13SvKjJIuS3J7k2CTrrY66pLVVkjcm+WmSe5M8nORXST6eZP3VsC//frXKkmzV/r+mkmy8Grbv51Q955kDre2OBG4BPgTcA+wPnJlkYlV9aZj3tXO7/dnAmIE6JZkAXAbcBBwIbA+cRBPGPz7EfX4/SYAfAjOq6rGVqFtaUz0DuBz4HHAf8EJgBvBM4G+HeV/+/Wo4fA54ENhoNW3fz6l6zvscaK3WhoB7+rWdCbykqrYbYJ3dgLuran6HZfsDF1fVkg7LRlXV0vb3s4GJVbVnh35/D3yM5jblC9u2j9Ee9PRp+zmwTYcSL66qtyR5dlXNb7+d+ibwn1V1/EDvhbQuSPJp4H3AhOrwPyj/ftUrSV4BfB/4DE1I2KSqHhygr59TrbUcVqS1Wv9g0PovYMtBVpsBXJpkYt/GJEfQ3IVwrwH2tbTLsvaj+Yd3YZ+27wIbAtP6bG/3qprY4fGWdvn89ueDwOnAHl3uX1qb3QsMNqxoBv79aoS1w3W+BBxLc5Z6RWbg51RrKcOB1kUvAf5nkOXTgceAi5KMA0hyIPB14JiqunQV9z8FmNe3oap+S3NFiCndbCDJRn1qGw0cDMxdxbqkNVKS9ZKMTfIy4P3Av3Q6a9Cajn+/GnnvAZ4GfLnL/tPxc6q1lOFA65QkewOvpxl72VFVLQD2BSYAP2hP8Z4FnFRVJwxDGRNoxk/3t6Bd1o0tgP9IMhe4HlgCfHoYapPWRA+1j58AVwIfHaijf78aaUmeAfwDcGS34/H9nGpt5oRkrTOSTALOBGZV1czB+lbVHUn2Aa4Gzge+VlVHre4au1VVvwF27XUd0gjZAxhLMyH5k8DJwHsH6uzfr0bYp4HZVXXBUFbyc6q1leFA64QkTwcuBG4D3tLlahvSjG1eCmzcdyLYKloAjO/QPqFdJqmPqvp5++tVSe4BvpHkpKq6eZDV/PvVapdkZ+AdwCuSbNo2j21/jk+ypKoWD7IJP6da6zisSGu9JGNpLsG2PvC6qlrUxTqTgEtoLhc3DXgd3Y8lXZF59BvzmWRrmv+hzOu4hqRllgWFjlcbA/9+NaJ2oLmk6DU0B90L+NNnbT7NJOWO/JxqbeWZA63V2klU36P5B3yPqrqri3W2AC4FbgYOqarF7USxC5IsqKpjVrGsC4GPJtmkqh5o2w4DFtOMp5Y0sJe2P2/ptNC/X42wq4BX9mt7DXAUzf0IftNpJT+nWpt5nwOt1ZJ8Ffgb4APAf/Zb/F9V9UiHdX5Ic5OlvfpeBq79h/tsmrMPF3dYbyzN/wwAPgyMA/5f+/yCZWcs2pvT3AT8AjgBeA7weeCLVTXUm9NI66wkF9HcyOlGmsmQL6X52/phVb1pgHX8+1VPJZkOnMHg9znwc6q1luFAa7UktwLbDrB4u6q6tcM6zwX+2OkeCUn2AK4Z4OZLkxjg28z++0qyE82kypfQXFHiazR3nnzSTW+kp6ok/wC8AZgEPE7zLewZwCkDXRXGv1/1WpfhwM+p1lqGA0mSJEmAE5IlSZIktQwHkiRJkgDDgSRJkqSW4UCSJEkSYDiQJEmS1DIcSJIkSQIMB5IkSZJahgNJeopJMjPJnF7X0V+SSvK3va5jqJK8OskHe12HJA0Hw4EkaU3xEuB7vS5iJbwaMBxIWieM7nUBkqR1U5INq2pxt/2ravbqrGcohlq7JK0rPHMgSU9xSbZJ8t0kf0yyKMnFSZ7Xr89nk9yQ5MEk85N8O8kz+/W5NclJST6RZD6wsG2fmWROklclmZvkoSRXJdm53/rLDStKckWSs5O8OcmvkyxMcmGSZ3eo/8Iki5PckmR6u94VQ3gPKsmRSb6Y5G7ghrb9tUkuTXJXu//ZSV7dZ70ZwIeBbdttVJKZfZa/PMmV7ft6b5LTkmzSbV2SNNI8cyBJT2FJng5cBdwLvAdYBBwNXJbkuX2+Pd8c+AxwO7AZzQHx5Un+rKqW9tnkm4Ebgfey/P9jtgE+B3z6/7d3byFWVlEAx/8rjbIoswtKWhJUPkRU1FhpaIUvXiCsyIcKwl56rCAjwuzykERlkRVEdKGHSITCQKOSKBwLhx4KC0kKI0UjMVMib7l62PvQ+DFzdJrGmXH+PxjOOd/59j5r9st863xr7QH+Ap4F3ouIyzMz24R4LXB+/bwxwIvAa8CcGn8Aq4CzgIXAPmBxjfHHPi7HQ8AXwN38++XZRcCHNd7DwGxgTUTMyMxO4HXgEuBmYH4d81uNbTrwKfABcDtwDrAUGFdfS9KQY3IgSSPbA8DpwJWZuQsgIjqBLZSL7ZcBMnNha0BEjAK+BLYCN1AuqLubl5n7GsfOBqZn5uY6x0nA+8AUYFOb+M4E5mbm73XcBGBZt7KfOcAVwNTM7KrnbKjx9zU52J6ZC7ofyMzlrec15s+Ay4B7gc7M3BoR24H9PZRFLQXWd58zIrYBa2tStbGP8UnSgLOsSJJGtlnAJ8CeiBgdEaOBvcDXwDWtkyJidkSsj4g/gEOUxADg0sZ8a3tIDAC2tBKD6vv6OKmHc7vraiUGjXET62MHsKOVGABk5rYaf1+tbh6IiEkR8Xa9qD8EHKQ0IDd/7+a40ygN1ita61rXdl2d4+r/EJ8kDTiTA0ka2c4FFlAuWLv/3ARcABARHZTSna2Ukpvrgevq+FMb8/3ay+fsbrw+0Mv4vo6bQC3jaejp2NEcEXu9U7AKmAY8RlmTDmANR497HDAKeIUj13U/cDJ1bSVpqLGsSJJGtl2UC+Cnenhvb32cT7nYXtDqD4iIyb3M165/YCDsoPQXNJ1H6T/oi2bsFwNXAbMz86PWwYgYcwxz7a7zPU4PdyQovRuSNOSYHEjSyLYWuAP4rs3WnWOAg43G4TsHPLJj0wUsiYipmbkBICImUsp2Ovs5dysJ2N86UJOi6cC33c47QONOQmb+GRFfAVMy88l+xiFJx43JgSSNbM8Dd1F2HnoJ2AaMB2YC6zLzXUpPwv0R8QJl555pdcxQsBr4hlLb/whlJ6QllBKhw+0GHoNNlFKq5yJiMXAG8ARljZrnjY+Ie4CNwM7M3AIsojQfHwZWUu7EXAjMBR7NzB/6GZ8k/e/sOZCkESwzd1L6BzYBy4CPgWeAsdRvxzNzNfAwcBulBGkmMG8w4m2qdzNuocT/JmWr01cpjct7+jn3fuBWSiPySkrp1dPA541TVwBvUdati1JKRGauA2ZQSpzeoSRWi4Bf6L03Q5IGVbTfXlqSpOElIsYCPwHLM3PJYMcjScOJZUWSpGEtIu6jlBBtpnxL/yBwCvDGYMYlScORyYEkabjbRyl7mkzZIWgDMCszfwao/1+gN5mZfw98iJI0PFhWJEk6oUVEuz90n2fmjccrFkka6rxzIEk60XW0eW9vm/ckacTxzoEkSZIkwK1MJUmSJFUmB5IkSZIAkwNJkiRJlcmBJEmSJMDkQJIkSVL1DzQUi/voqhJvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x504 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = 'learning_rate'\n",
    "y = 'epsilon'\n",
    "z = 'FinalObjectiveValue'\n",
    "\n",
    "scatter(data=results, x=x, y=y, z=z, title=x+' vs '+y, logx=True, logy=True, label=z, alpha=0.4, size=40);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Not enough iterations to generate plot (need at least three).\n"
     ]
    }
   ],
   "source": [
    "if (np.min([results[x].size, results[y].size]) >= 3):\n",
    "\n",
    "    surface_plot(x=results[x], y=results[y], z=results[z], xlabel=x, ylabel=y)\n",
    "else:\n",
    "    print('Not enough iterations to generate plot (need at least three).')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze categorical hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Could not interpret input 'optimizer'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-14-e872eb434029>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mboxplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'FinalObjectiveValue'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'optimizer'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mresults\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m;\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/envs/python3/lib/python3.6/site-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36mboxplot\u001b[0;34m(x, y, hue, data, order, hue_order, orient, color, palette, saturation, width, dodge, fliersize, linewidth, whis, notch, ax, **kwargs)\u001b[0m\n\u001b[1;32m   2229\u001b[0m     plotter = _BoxPlotter(x, y, hue, data, order, hue_order,\n\u001b[1;32m   2230\u001b[0m                           \u001b[0morient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcolor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpalette\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msaturation\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2231\u001b[0;31m                           width, dodge, fliersize, linewidth)\n\u001b[0m\u001b[1;32m   2232\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2233\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0max\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/python3/lib/python3.6/site-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, x, y, hue, data, order, hue_order, orient, color, palette, saturation, width, dodge, fliersize, linewidth)\u001b[0m\n\u001b[1;32m    444\u001b[0m                  width, dodge, fliersize, linewidth):\n\u001b[1;32m    445\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 446\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mestablish_variables\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morder\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhue_order\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    447\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mestablish_colors\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcolor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpalette\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msaturation\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    448\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/envs/python3/lib/python3.6/site-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36mestablish_variables\u001b[0;34m(self, x, y, hue, data, orient, order, hue_order, units)\u001b[0m\n\u001b[1;32m    153\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstring_types\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    154\u001b[0m                     \u001b[0merr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Could not interpret input '{}'\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 155\u001b[0;31m                     \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0merr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    156\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    157\u001b[0m             \u001b[0;31m# Figure out the plotting orientation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: Could not interpret input 'optimizer'"
     ]
    }
   ],
   "source": [
    "sns.boxplot(x='FinalObjectiveValue', y='optimizer', data=results);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get per-training job metric data\n",
    "We use the very convenient `TrainingJobAnalytics` from the SageMaker Python SDK to gather metrics collected by individual training jobs.\n",
    "\n",
    "<font color=\"brown\"> **SageMaker Cloudwatch metrics are stored for 2 weeks. This part of the analytics can only be run less than 2 weeks after the hyperparameter tuning job ran**\n",
    "</font>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def job_metric(jobname, metric_name):\n",
    "    jobdf = TrainingJobAnalytics(\n",
    "        training_job_name=jobname,\n",
    "        metric_names=[metric_name]).dataframe()\n",
    "    \n",
    "    jobdf['TrainingJobName'] = [jobname]*len(jobdf)\n",
    "    \n",
    "    return jobdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pick one of the following metrics for the next cell:\n",
      "   - train:loss\n",
      "   - train:accuracy\n",
      "   - validation:loss\n",
      "   - validation:accuracy\n"
     ]
    }
   ],
   "source": [
    "print('Pick one of the following metrics for the next cell:')\n",
    "\n",
    "samplejob = TrainingJobAnalytics(training_job_name=results['TrainingJobName'][0])\n",
    "metrics = samplejob.dataframe()['metric_name'].unique()\n",
    "print('\\n'.join([f'   - {metric}' for metric in metrics if metric != 'ObjectiveMetric']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "allmetrics = pd.concat([job_metric(job, 'validation:accuracy') for job in results['TrainingJobName']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>metric_name</th>\n",
       "      <th>value</th>\n",
       "      <th>TrainingJobName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>0.25</td>\n",
       "      <td>tensorflow-training-200612-1046-001-d6feee4c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>240.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>0.25</td>\n",
       "      <td>tensorflow-training-200612-1046-001-d6feee4c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>0.25</td>\n",
       "      <td>tensorflow-training-200612-1046-002-cc196201</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>240.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>0.25</td>\n",
       "      <td>tensorflow-training-200612-1046-002-cc196201</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   timestamp          metric_name  value  \\\n",
       "0        0.0  validation:accuracy   0.25   \n",
       "1      240.0  validation:accuracy   0.25   \n",
       "0        0.0  validation:accuracy   0.25   \n",
       "1      240.0  validation:accuracy   0.25   \n",
       "\n",
       "                                TrainingJobName  \n",
       "0  tensorflow-training-200612-1046-001-d6feee4c  \n",
       "1  tensorflow-training-200612-1046-001-d6feee4c  \n",
       "0  tensorflow-training-200612-1046-002-cc196201  \n",
       "1  tensorflow-training-200612-1046-002-cc196201  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "allmetrics.sort_values(by='value', ascending=not is_ascending).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add an \"Is_best\" marker\n",
    "allmetrics = pd.merge(allmetrics, results[['TrainingJobName', 'FinalObjectiveValue']], on='TrainingJobName')\n",
    "allmetrics['is_best'] = [1 if j == best_job else 0 for j in allmetrics['TrainingJobName']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvoAAAHmCAYAAAAC4yqoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xm8HFWd9/HPjxggwYQksq8RcIhsA3oVeMSBkZ0ZZFEEFJ0Iw/KIG7giIBEURQQU0WFRNtFHETMwCIhsCaKjEBQXIOxhCWKCJgSyQEh+zx9VN1Q6fe/te3Nzl8rn/Xr1q9OnTlWd6tN98+3qU6cjM5EkSZJUL6v0dwMkSZIk9T6DviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQl9SSiBgbERkRE/q7Lb0pIi6PiBUyz3Bdn7OVRURMi4hJ/d2OVpSvs8uXY/0J5TbG9lqjetaO5XrOB1OfSX3BoC8NABGxW/mfbEbEBR3UWSciXinrTFqOfY2PiE/2uLFaShnmJ0TE9v3dFg08vt8k9afX9XcDJC1lAfD+iPhUZr7csOyDQACvLuc+xgNjgW92c70ngWG9sP+6GQucBkwD7mtY5nOm8fTs/dZdw4BFy7H+l4GvAY1/d/raloC/5Cn1Es/oSwPLfwOjgQOaLPswcCN9/B9xRIwAyMKCzDS0tsjnrGNReH1/t2OgaX+/dVf5OlvY0/1m5qvlNvo1ZGfmy5n5Sn+2QaoTg740sPwe+BNFqF8iIt4ObA1c1tGKEdEWEf8dEc9HxMsR8VBEnBwRr6vUmQbsCmxaGSqUEbFbuXxSOcZ1s4i4JiL+Acwpl3U43jwi3lOuOzsi5pX7Pj8iVu3qgCNizYg4KyIeLds9MyL+X0RsVqmzb7nvj3ewjf8t1xtaKfuXiLglIl6IiPkR8fuIOKqr9lSfhyblSz0HETEeuKNcfFnl+ZzUrH5lO6+LiM9FxAMRsSAi/l723bYd7S8i/j0i7inr/zUizq72bQvH9J/lczC/fE5+GRG7VJYPiYhnI+L3Hax/bNmWAytlq0XEFyLi/rJdsyPi+ojYoWHd9qFp4yPi+Ih4gOLbq0+30O49yrbOLvfxp4g4rkm9vSLiJxHxeHmMs8v1du1gu1tExGUR8UwUQ+KejYjrIuKtTeqOi4gbIuLF8rm7JiLWa6Ht0+j5+22VKN6/d0bEc2Ubn4qI/4qINzTZ1zJj9NvLImLniJgcEXPL19r3ouFDVjQZo18p2zIiziyfq5cj4o8RsV+TNgyPiHPL1+f8iPhtROwe3bgOJjoYYx8RB0bEr8tjeKn8d7MTIu313xIRt5d1/xERV0TEOq20QaoTh+5IA8+lwLkRsWFmTi/LjgRmAD9vtkJE/BswEXgUOAf4B7AzcDqwPXBIWfWTwFeBtYATKpt4sPLv1wOTgV8DJwOd/ucYEV8BvgA8AJwH/BXYHHgP8EWgw7NzEbEm8Btgk/K47wfWBz4C/C4i2jLzSeCXwHPAh4DzG7bxJmAn4Pz2M5oRsT/FtyPPlc/Hi8BhwPciYrPMPLmzY+qGO4EzKY7/YuBXZfnfuljvh8D7gFuA/wLWA44H/jci3pmZf2iovx/Fc3IhxfN0AEVInlXuv1MRcRbwWeDusq0jgGOAOyLigMy8MTMXRcRVwGciYuvMvL9hMx8CngduKLc5FPgF8H+AHwAXAGsCRwO/joh/ycwpDdv4JPAG4BKKvnm6i3YfUx7zb4GvAHOBPYH/iojNM/MzlerjgTHAlcAzwIbAfwK3RcS/ZuavKtttA24DhgLfB/5SrrtreTz3Vra7ITCJ4vX0GeCfgWOBkcBenbWf5Xu/rVru72fAdeWxvw04CtglIt7a4pnv7Sn+blwG/AjYrdzGYorXQCuuABYC3yjb9Ung2oj4p8ycVqn3U4rX6rXArcAbKZ63J1rcT1MR8RHgO8BUir9pUPT3tRFxbGZe3LDKRhT9+zPgGuAtFH9D2yLibZk5b3naIw0qmenNm7d+vlH855sU4e0NFMNzvlAuGwbMBr5RPn4JmFRZd3WK0HQn8LqG7Z5Qbne3StkkYFoH7ZhU1v9yk2Vjy2UTKmVvL8tuB1ZvqB9AdHHc3wLmA//cUL4pxZnNyytlZ5f72qqh7hll+VvKx0MoxsbPBjao1FuVIkwtAt5UKb+8+FO4zPOwzHPUwXPQ3nfjW6y/Z1n2k+rzQxEgXwV+1WT9ucDYhuf2L8BfW3htbUkR6u4CVq2Ub1A+R9OAIWXZ1uX+vt6wjc3L8vObvLb2bqg7Eniq4TXa/hz9A1inxffE+hRn/X/UwetmEbBZpWyNJvXWpfhwcmOT524BsF2TdVap/Hta2e73NdT5Tlm+ZQvH0fS11ML7LYBhTcqP6qBNSeX9UilbDOzYUH4DRXB/faVsQll/bJOynze8Vt9Wln+1UrZfWXZJw77ay7PxWDp4TqY1vHZGU/zNexQY2fA6e4ziQ/yoJn32yYbttr9eP99KO7x5q8vNoTvSAJOZfwf+h+KMFcDBFGdKL+1glT0pAs1lwKiIWKv9RjGmH7o+89joGy3W+0B5f1JmLqguyFJHK0ZElOvfCUxvaPdcirO41XZfUd5/qGEbRwB/ycz2ISdvpfyGIDOfrbTnFeDrFEMWO/zKvw8cVN5/pfr8ZOYfgespztau3bDOtVk5c1qudwewXuMQjCYOoAiNX8/KGeDyubmM4kPVDmXZ/RRnsz8QEdX/H9qf8ysqZUdQnGG9t6HvVqX4pmKXiBjW0JYrM3NGF+1t915gNeD71e2X+7ieoh/3qBzP3PZ/R8Try+Eti4DfATtWtrs95TC4zPxT404zc3FD0bOZeXVD2e3l/ZtaPJauLPN+K98+82HJsKpR5bG373vHxnU68L+Z+buGstspvtEf2+I2vtXwWr2HInxXj3//8v7c6oqZeSNLf4PRXXsCa1B8yJxT2e4cim/3Xk/ldVCaA3y3oey7ZflBSCsRh+5IA9NlwA1RjKE+Erg7Mx/ooO6by/uOPghA8UGgVTMzc3aLdd9EcZbsj93Yfru1Kb692AuY2UGdJaErM/8SxfjxD0TEF8pA9i8UYeWzlXXeWN43Dj2plm3WZFlfeSPFcTULP/cDB5Z1qs/J403q/r28fwNF6Opsf+3bbrY/KJ6P9mE2V1AEqD2AX1Y+TN2fmdUhLW+m+Lapo76DYshKdXjOw53UbdT+ur61kzpLXtcRsTnF8J69gVEN9aofONvDaePwqI509dwvrw7fbxHxPuBTFB/EhjYsHt3i9nuj/R1to7p+++v60SZ1H+K1/uyunryfH8+GYU2Z+XJEPN6krlRrBn1pYLoZmE4xbeO/Av+3k7pR3n+GZad3bPdsB+XNdHf8atKz6fDa230rcFaL61xJMU3hu8r1PkRx1vaqHuy/Mx0dT3/9zexs2sToZFlP/D+K6xo+RHFtxC4U4ehzTfb7Z+DETrbV+CGgO6+t9uP6EMV1H808DsUZfIpvhtageH38mWJIx2LgJIrXS0+t6Oe+6XMSEQdTDO+6G/gExQemBRRD035B65Np9Eb7O9pGs/WdGlMaQAz60gCUxYWRV1KElPkU4asjj5T3czOzs7OfSza/vO2reBjYl2J8+d3dXHcmxRjxkS22G4qLCc8GPhQRv6YY3nFLZlaDYPvZx62brL9VQ52O/INiCFCjZmcDu/t8Pk4R0t5MMcNSVXv7luvixSb7g+L5eKyD/S15PjLz+Yi4ETioDNAfogjMjR+mHqH4Vub2JsNdekP76/r5Fl4fu1Ncc3BkZi41M1VEfLmhbvu3Cn31A2c9fb99kCLY/2tWLh6NiHG90qreN43idf0mlv22asvl2G719Xtbw7KO3s+bRcSq1bP6EbEaxft36nK0RRp0HKMvDVwXAl8CjquOTW3iZooZeT4fEWMaF0bEsFh6bu6XgNHlkIzl9aPy/sxoMpVmZ/sow+EPgbdHxHub1WmcDi8zZwI3UVy38AGKC/KuaFjt9xQXg344KlMglrPEfIYieF3X+WHxMDAiimlN29dfhaVnTmnXPmxmmee+A9eW9ydVn5+I2AZ4N3BXeZy95X8ojvkzsfT0o+tTTOP6JMsOY7kCGE4xZOcQig9Tjd8KXUkxW1DTM/oR0Z3hYs1cTXFR+peajPVvn5Z1tfJh+xnnaKizF8uOZf8jxZCPIyNimQ+DvfS+qOrp+20RRb8t+X+63MYpvdi23nR9eb/Ue6SchrOnw3aguN5jLvCx6t+x8t8fo3h+b2lYZyTFLFVVHynLr0VaiXhGXxqgMvMpilkvuqo3NyI+RPEf2EMRcSnFONlRwDiKUHwQxQwfUFzk+u/ABRHxG4pAcXs3LpKs7vvucurGzwG/j4ifUMwA9EaKs+1vpzhr35GTgXcAV0fE1WXbXqG4QHQ/igtDxzescwVFID4HeIGG/7jLb0M+SjGt3z0RcTHFMI5DKabhPDMzH6FzF1OMjf7viPhW2ab30vxv5gPl9j8SEfPK452Rmbc3qUtm3lIe62EUAfDnvDa95gKg6W8F9FRmPhQRZ1Ncx3Bn2Uft02u+HvhAZjYOzbiBYgz2WTT/MAXFzDd7AmdHxLsoLvCcQ3Eh9O7lsfzrcrT7mYj4v8D3gAcj4gcUH0rWBraluJZhK4ozyXdRTqUaxTzwz1Ccsf8gxTCebSvbzYj4MMXZ4bsjon16zVEU02v+Avh2T9vdRE/fb9dQTFF7e/nt3lCKYx7ei23rTTdSnHQ4urxouH16zWMovrnaricbzczZEfFZipmOfhev/VbAeGAL4NjMfKFhtceA08oPz/dSfDt3JMXZ/PORVib9Pe2PN2/elp5es4W6S02vWSnfhmJ4xXSKYPo3ijnqTwXGVOoNp5g7/G+8dtZwt3LZJDqeCnAsDVNFVpYdTjF15YsUZ9+mUoyVXrWF4xletvHPFMOUXqT46v8SGqYFLOuvShFCl5nKr6HerhRn+uZQhM4/AEc1qXc5Tab+o/igcR/FWeVnKULvls2eg7Lu78v9ZHv/dPScUXxg+Fx5nC9TDBW6Fti2G8/5BBqmQ+zieT66fA4WlM/JLcA7O6n/7XL7L9BkmsfKcXwcuKfs97kUQ25+COzV5PU9vgfvjXdQfGibUb6un6WYcehTVKZ0pQiSv6D4bYEXy9fyOzvp3y0p3i/PVbZ7LeU0rWWdaTR/r7V8PPTw/Vbps/YfF/srxQfQMXQ8lWaXZWX5+Go7Ono9dfYaa/bc8No1En+jeC//juL6iGuAeS32d0fP+UEUf8/aX2e/AQ7saH2KufNvL+vOovith3W7+/rz5m2w3yLT62YkSdKKERF/BoZmZpfXF0TE08BDmdk4ZaakHnCMviRJWm4dXEvxbxTfNjaOo2+2/qoUU7J2exihpOYcoy9JknrDFyNiB4qhVS9QXCdxJK9d79Gh8oL8Ayh+6fvmFdxOaaXh0B1JkrTcyhl2Pk9xkfSaFNee3A6cmpnNfkiruu7jFCcfL6e4JmVFTNkqrXQM+pIkSVINOUZfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrodf3dgMFmrbXWyrFjx/Z3MyRJklRz99577/OZuXZP1zfod9PYsWOZMmVKfzdDkiRJNRcRTy7P+g7dkSRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg05vaYkSVINzZkzhxkzZrBw4cL+booaDB06lHXWWYeRI0eu0P0Y9CVJkmpmzpw5/O1vf2PDDTdk2LBhRER/N0mlzGT+/PlMnz4dYIWGfYfuSJIk1cyMGTPYcMMNGT58uCF/gIkIhg8fzoYbbsiMGTNW6L4M+pIkSTWzcOFChg0b1t/NUCeGDRu2wodVGfQlSZJqyDP5A1tf9I9BX5IkSaohg74kSZIGnPHjx9PW1tYr24oILrjggl7ZVmcefvhhJkyYwOzZs1f4vlph0JckSdKAc+qpp3L55Zf3dzO65eGHH+ZLX/rSgAn6Tq8pSZKkAWfzzTfv7yYMep7RlyRJ0oBTHboze/Zs/vM//5MNNtiA1VdfnU022YSjjz66W9t75ZVX+MQnPsGYMWMYNWoUH/vYx3jllVeWqvPUU09x2GGHMWbMGIYPH87ee+/NQw89tFSdr371q2yxxRasvvrqrLvuuuyzzz4899xzTJo0if333x+AN77xjUQEY8eO7fkT0As8oy9JkqQB7cQTT+Q3v/kN5513Huuttx5PP/00d955Z7e2cc4557DTTjvxwx/+kPvvv5+TTz6Z1VdfnbPPPhuAf/zjH+yyyy684Q1v4MILL2T48OF87WtfY4899uDhhx9m2LBhXHnllZx55pmcddZZbL311vz973/n9ttvZ+7cubzlLW/hG9/4Bp/+9KeZOHEi66+/PqutttqKeDpaZtCXJEnSgHb33Xdz/PHHc+ihhy4pO+KII7q1jREjRvDTn/6UVVZZhX333ZeXX36Zr3zlK5x00kmMGTOG8847j7lz53LfffcxZswYAN7xjncwduxYLr30Uo4//njuvvtu9tprLz7ykY8s2e7BBx+85N9bbrklADvssEO/n80Hh+5IkiRpgNt+++05++yz+e53v8vDDz/co20ccMABrLLKa9H34IMPZv78+fzlL38B4NZbb2XPPfdk5MiRvPrqq7z66quMGDGCt771rUyZMmVJO2688UZOO+007r77bhYtWrT8B7cCGfQlSZI0oF1wwQUceOCBnH766Wy55Za86U1v4sc//nG3trHOOus0ffzXv/4VgOeff56f/OQnDB06dKnbHXfcwdNPPw3AkUceyZlnnsnVV1/NjjvuyLrrrsspp5wyYAO/QV+SJEkD2qhRozj//PN57rnn+OMf/8iOO+7IBz7wAR544IGWtzFjxoymj9dff30AxowZw7vf/W7uueeeZW7f+c53AFhllVU44YQTePDBB3nqqaf49Kc/zVe/+lUuueSSXjrS3mXQlyRJ0qCx3XbbcfbZZ7N48WKmTp3a8nrXXXcdixcvXvJ44sSJDBs2jG222QaA3Xffnfvvv5+tt96atra2pW7tY++rNt54Yz7/+c+zxRZbLPnAseqqqwKwYMGC5TnEXuPFuJIkSRrQdtllFw466CC22WYbIoJLLrmENdZYg7e//e0tb+PFF1/kkEMO4eijj+b+++/njDPO4Pjjj19y4e2JJ57IVVddxbve9S4+9rGPseGGG/K3v/2NyZMns8suu3D44Ydz7LHHMmbMGHbaaSfWXHNN7rjjDh555BHOOuss4LWLcS+66CIOO+wwhg8fzrbbbtv7T0iLDPqSJEka0HbeeWcuv/xypk2bxpAhQ9hhhx246aab2GijjVrexqc+9Skef/xxDj/8cBYvXsxRRx3FmWeeuWT5WmutxW9/+1tOPvlkTjjhBGbPns3666/PLrvswnbbbbekHZdccgkXXXQRCxYsYIsttuCSSy7hwAMPBGDTTTflG9/4Bueffz7f/va32WijjZg2bVqvPhfdEZnZbzsfjNra2rL9ymtJkqSB6MEHH+TNb35zfzdDXeiqnyLi3sxs6+n2HaMvSZIk1ZBDdyRJkjRovfrqqx0uiwiGDBnSh60ZWDyjL0mSpEGrcd776m333Xfv7+b1K8/oS5IkadC65557Olw2YsSIPmzJwGPQlyRJ0qDV1tbja1Vrz6E7kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJkgakBx54gN13353hw4ezwQYb8MUvfpFFixb1d7MGDefRlyRJ0oAza9Ys9thjD7baaiuuu+46HnvsMT71qU+xePFivvzlL/d38wYFg74kSZIGnAsvvJD58+czceJERo4cyZ577smcOXOYMGECn/3sZxk5cmR/N3HAc+iOJEmSBpybbrqJvffee6lAf9hhhzF//nwmT57cjy0bPAz6kiRJGnCmTp3KuHHjlirbZJNNGD58OFOnTu2nVg0uBn1JkiQNOLNmzWLUqFHLlI8ePZpZs2b1Q4sGH8foS5IkrSTuu+8+Zs+e3ef7HTVqFNtvv32f73dlZ9CXJElaSQymsD169GheeOGFZcpnzZrF6NGj+6FFg0+fD92JiK0i4raImBcRz0bE6RExpIt13hYRl0XEo+V6D0XEaRGxepO6wyPirIh4KiIWRMTjEfHZhjqrRcQ5ETEjIuZGxA0RMbZ3j1SSJEk9NW7cuGXG4j/99NPMmzdvmbH7aq5Pz+hHxGjgVuAB4ABgc+Acig8cp3Sy6qFl3bOAR4DtgDPK+/dUtj8EuBFYDzgZeLpc7w0N2zsfeC9wAjATmADcEhHbZuaC5TlGSZIkLb99992Xs88+mxdffJERI0YA8JOf/IRhw4ax66679nPrBoe+HrpzHDAMODgz51CE65HAhIj4elnWzNcy8/nK40kRsQC4KCI2zcwny/KjgX8GtszMGe11qxuKiI2Ao4AjM/PKsuxPwBPAEcD3lvsoJUmStFyOO+44zj//fA4++GA+97nP8fjjjzNhwgROPPFE59BvUV8P3dkXuLkh0P+YIvx3+NGsIeS3+0N5v0Gl7Ejg6krIb2av8n5iZfvTgbvK9kmSJKmfjR49mttuu41Fixax//77c9ppp3HCCSfwpS99qb+bNmj09Rn9ccDt1YLMfCoi5pXLru/GtnYGFgOPAUTEqsAOwPUR8UPgYOAVikD/icqHi3HAM5n5UsP2HgR269bRSJIkaYXZaqutuP3227uuqKb6+oz+aKDZnE6zymUtiYj1KMb0/6By9v4NFB9cPgusAbwbOJHiWoDqcJxeaYMkSZI0kA266TXLM/dXAy9RXEy7ZFF5Pws4JDMXlvUXAldExOaZ+VgP93kMcAwUv8gmSZIkDXR9fUZ/FrBmk/LR5bJORUQAVwJbA/tlZnWd9rP0v24P+aX273u26mkbMvPizGzLzLa11167q2ZKkiRJ/a6vg/5UijHyS0TExsDwcllXvkkxFOeAzFyqfmbOA57ktTP7S3ZR3i+utGHjiFijod64FtsgSZIkDXh9HfRvAvaOiBGVskOB+cDkzlaMiJOAjwJHZOZdHVT7OfCOcnhPu90pQv6fy8e/LO8Pqmx7A+CdZfskSZKkQa+vg/6FwMvAxIjYoxz7PgE4tzrlZvkLuN+vPH4/cCbFsJ3pEbFT5VYdS3M2MAL4WUTsW27/PODSzHwKIDOfAb4PfDMiPhgR+1DMzPMkcNWKO3RJkiSp7/TpxbiZOSsidgcuoJhKczZFEJ/QpF1DKo/b574fX96qPgxcXm7/yYjYo9zmRGAOcAXw+YZ1Pg7MBc6lGDY0GTjcX8WVJElSXfT5rDuZ+QDwri7qjG14PJ5lA35H606hGIbTWZ2XKabePLGVbUqSJEmDTV8P3ZEkSZLUBwz6kiRJGpAeffRRjj32WLbbbjuGDBnCbrvt1t9NGlQG3Q9mSZIkaeVw//33c+ONN7LTTjuxcOHCrlfQUjyjL0mSpAFp//335+mnn+anP/0pW2+9dX83Z9Ax6EuSJGlAWmUVo+ry8NmTJEmSasigL0mSJNWQF+NKkiStBCL6uwWQ2d8tWLl4Rl+SJEmqIYO+JEmSVEMO3ZEkSVoJOGxm5WPQlyRJ0oA0b948brzxRgCmT5/OnDlzuOaaawDYb7/9GD58eH82b8Az6EuSJGlAmjFjBocccshSZe2Pn3jiCcaOHdsPrRo8DPqSJEkakMaOHUs65qjHvBhXkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiTVkLPVDGx90T8GfUmSpJoZOnQo8+fP7+9mqBPz589n6NChK3QfBn1JkqSaWWeddZg+fTrz5s3zzP4Ak5nMmzeP6dOns84666zQffmDWZIkSTUzcuRIAJ599lkWLlzYz61Ro6FDh7Luuusu6acVxaAvSZJUQyNHjlzhQVIDm0N3JEmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBrq86AfEVtFxG0RMS8ino2I0yNiSBfrvC0iLouIR8v1HoqI0yJi9YZ6EyIim9z2aajXrM5vV8TxSpIkSf3hdX25s4gYDdwKPAAcAGwOnEPxgeOUTlY9tKx7FvAIsB1wRnn/noa6LwD7NJQ92GSb5wDXVB6/2NJBSJIkSYNAnwZ94DhgGHBwZs4BbomIkcCEiPh6WdbM1zLz+crjSRGxALgoIjbNzCcry17NzFbOzk9rsZ4kSZI06PT10J19gZsbAv2PKcL/rh2t1BDy2/2hvN+g95onSZIk1UNfB/1xwNRqQWY+Bcwrl3XHzsBi4LGG8lER8XxELIyIP0TEwR2sPyEiXi3rXhoRY7q5f0mSJGnA6uugPxqY3aR8VrmsJRGxHsWY/h9k5ozKokeBzwKHUIzdfxb4WZOwfwVwLPAu4EzgIIphRE0vCo6IYyJiSkRMmTlzZqvNlCRJkvpNX4/RX24RsSpwNfAScEJ1WWZe1VD3euA3wBeBiZV64yvV7oyIB4Ebgf2Baxv3mZkXAxcDtLW1ZW8chyRJkrQi9fUZ/VnAmk3KR5fLOhURAVwJbA3sl5mdrpOZSRFWVxuTAAAeZUlEQVTwt+tiCs9fUHxweEtXbZAkSZIGg74+oz+VhrH4EbExMJyGsfsd+CbFtJx7ZmYr9QGyvHVcITOLzxCd15MkSZIGi74+o38TsHdEjKiUHQrMByZ3tmJEnAR8FDgiM+9qZWflNwDvAf6YmYs6qbcP8Hrg3la2K0mSJA10fX1G/0Lg48DEiDgL2AyYAJxbnXIzIh4FJmfmUeXj91NcNHs5MD0idqps87HMnFnWmwz8jOLbgTWAo4EdgQMr2z4GaKP44a7nKYbrnALcDdzQ60csSZIk9YM+DfqZOSsidgcuAK6nmIHnPIqw39iu6pj6vcr78eWt6sMUHwCgmHXnk8D6FFNv/h74t8y8qVL/MeA/KM70jwSeoxj3f2pnZ/0lSZKkwSSK61XVqra2tpwyZUp/N0OSJEk1FxH3ZmZbT9fv6zH6kiRJkvqAQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEMGfUmSJKmGDPqSJElSDRn0JUmSpBoy6EuSJEk1ZNCXJEmSasigL0mSJNWQQV+SJEmqIYO+JEmSVEOv6+4KETEa2AbYGLgpM2dFxOrAK5m5uLcbKEmSJKn7Wj6jHxFDIuLrwDPAZOAHwBvLxT8DTuv95kmSJEnqie4M3TkTOBr4KLAZEJVl1wH792K7JEmSJC2H7gzd+RDw+cy8LCKGNCx7jCL8S5IkSRoAunNGfxRFoG9mVaAx/EuSJEnqJ90J+n8BDuhg2b7A75e/OZIkSZJ6Q3eG7nwZ+FlEDAN+CiSwfUQcBBwLvHsFtE+SJElSD7R8Rj8zrwPeD+wB3ERxMe73gPHABzPz5hXRQEmSJEnd16159DPzauDqiPgnYC3gH8BDmZkronGSJEmSeqbbP5gFkJkPAw/3clskSZIk9ZKWg375Y1mdyszPLl9zJEmSJPWG7pzRP6RJ2WhgJPACMAsw6EuSJEkDQMtBPzPf2Kw8InYELgaO661GSZIkSVo+3ZlHv6nM/B1wNnBBK/UjYquIuC0i5kXEsxFxepNf2m1c520RcVlEPFqu91BEnBYRqzfUmxAR2eS2T0O91SLinIiYERFzI+KGiBjbvSOXJEmSBq4eXYzbxN+BLbuqFBGjgVuBByh+fGtz4ByKDxyndLLqoWXds4BHgO2AM8r79zTUfQHYp6HswYbH5wPvBU4AZgITgFsiYtvMXNDVcUiSJEkDXXcuxh3epHhV4M3A6cD9LWzmOGAYcHBmzqEI1yOBCRHx9bKsma9l5vOVx5MiYgFwUURsmplPVpa9mpm/7eQ4NgKOAo7MzCvLsj8BTwBHUPw2gCRJkjSodWfozkvAiw23vwO/BtYDPtLCNvYFbm4I9D+mCP+7drRSQ8hv94fyfoMW9lu1V3k/sbL96cBdZfskSZKkQa87Q3eOBBp/GGsB8Axwd2YubGEb44DbqwWZ+VREzCuXXd+N9uwMLAYeaygfFRHPA2sCfwHOyMyJleXjgGcy86WG9R4EduvG/iVJkqQBqzuz7lzeC/sbDcxuUj6rXNaSiFiPYkz/DzJzRmXRoxRTfP4BGAEcC/wsIt5TCfu90oa+FtHfLZAkSVp5ZePp7kGgty7G7TMRsSpwNcVQohOqyzLzqoa61wO/Ab5IZahOD/Z5DHAMwCabbNLTzUiSJEl9ptOgHxEzWXa4Tocyc50uqsyiGFLTaHS5rFMREcCVwNbAOzKz03UyMyNiInBWRAzJzEU9aUNmXkzxWwG0tbUNws9zkiRJWtl0dUb/O3Qj6LdgKsUY+SUiYmNgeLmsK9+kmJZzz8xspT4U7a8ew1Rg44hYIzPnVsrHtdiGfjEYvy6SJElS/+k06GfmhF7e303AZyJiRGa+WJYdCswHJne2YkScBHwUeF9m3tXKzspvAN4D/LE8mw/wy/L+IOCqst4GwDtpbeYgSZIkacDr6zH6FwIfByZGxFnAZhQ/VnVudcrNiHgUmJyZR5WP3w+cCVwOTI+InSrbfCwzZ5b1JgM/ozgzvwZwNLAjcGB75cx8JiK+D3yz/CDQ/oNZT1IGf0mSJGmw61bQj4idKX5s6p+A1RuXZ+bbO1s/M2dFxO7ABRRTac4GzqMI2o3tGlJ53D73/fjyVvVhig8AUMy680lgfYqpN38P/Ftm3tSwzseBucC5FMOGJgOH+6u4kiRJqovIFgd/R8SewI3AbRTB+yaKH7p6B8Vc+pMz88gV1M4Bo62tLadMmdLfzZAkSVLNRcS9mdnW0/W788u4pwPfAv6tfHxqZr6L4uz+QmBSTxshSZIkqXd1J+hvRXEWfzHFLDZrAGTmkxRDb07u7cZJkiRJ6pnuBP0FwCpZjPX5K7B5ZdkcYKPebJgkSZKknuvOxbh/BLYEbqEYp39SREwHXqEY1vPn3m+eJEmSpJ7ozhn9b/LaD099gWLWmpuBO4B1gON7t2mSJEmSeqo7Z/QXAN8FyMzpEfFWYAuKmXemZuYrK6B9kiRJknqgO0H/VuC5iPgp8JPM/A3wyIppliRJkqTl0Z2hO9sC3wP2Bu6KiCcj4uzyzL4kSZKkAaTloJ+Z92fmFzNzHPAW4EfAQcA9EfFoRHx5RTVSkiRJUvd054z+Epl5X2aelJlbAO+mGKd/Uq+2TJIkSVKPdWeM/hIRMRp4D3AosCswn+IMvyRJkqQBoOWgHxEjKYbqHArsDrwK3AAcBtyYmQtWSAslSZIkdVt3zujPBBZTzJ0/HvifzJy7IholSZIkafl0J+gfA1ybmS+sqMZIkiRJ6h0tB/3MvGJFNkSSJElS7+nRrDuSJEmSBjaDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqqE+D/oRsVVE3BYR8yLi2Yg4PSKGdLHO2yLisoh4tFzvoYg4LSJW72SdHSJiUUQ832RZNrn9tjeOT5IkSRoIXteXO4uI0cCtwAPAAcDmwDkUHzhO6WTVQ8u6ZwGPANsBZ5T372mynwAuAGbS8TGeA1xTefxiNw5FkiRJGtD6NOgDxwHDgIMzcw5wS0SMBCZExNfLsma+lpnVM/OTImIBcFFEbJqZTzbUPwJYF7gUOKaDbU7LTM/iS5IkqZb6eujOvsDNDYH+xxThf9eOVmoI+e3+UN5vUC2MiBEUZ/4/DbyyXK2VJEmSBqm+DvrjgKnVgsx8CphXLuuOnYHFwGMN5V8EHszMa7tYf0JEvBoRz0fEpRExppv7lyRJkgasvh66MxqY3aR8VrmsJRGxHsWY/h9k5oxK+ZbA8cCOXWziCuB6ijH8bcCpwD9HxNszc1Gr7ZAkSZIGqr4O+sstIlYFrgZeAk5oWPwt4PLM/HNn28jM8ZWHd0bEg8CNwP7AMt8ERMQxlGP9N9lkkx63XZIkSeorfT10ZxawZpPy0eWyTpWz6VwJbA3sl5mzKsv2Bd4BnBsRoyJiFLB6udqoiFitk03/guKDw1uaLczMizOzLTPb1l577a6aKUmSJPW7vj6jP5WGsfgRsTEwnIax+x34JsW0nHtmZmP9LYHXU0y/2WgWxfCcLzfbaGZm8RmCbKENkiRJ0oDX10H/JuAzETEiM9vnrT8UmA9M7mzFiDgJ+Cjwvsy8q0mVa4D7GsrGAwdRfDh4opNt70PxIeHeFo5BkiRJGvD6OuhfCHwcmBgRZwGbAROAc6tTbkbEo8DkzDyqfPx+4EzgcmB6ROxU2eZjmTkzM58BnqnuLCJ2AxZm5qRK2TEUF+DeCjxPMVznFOBu4IZePFZJkiSp3/Rp0M/MWRGxO8Wv1l5PMQPPeRRhv7FdQyqP9yrvx5e3qg9TfABo1WPAf1D8ou5I4DmKcf+nOuOOJEmS6iIyHZbeHW1tbTllypT+boYkSZJqLiLuzcy2nq7f17PuSJIkSeoDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqiGDviRJklRDBn1JkiSphgz6kiRJUg0Z9CVJkqQaMuhLkiRJNWTQlyRJkmrIoC9JkiTVkEFfkiRJqqE+D/oRsVVE3BYR8yLi2Yg4PSKGdLHO2yLisoh4tFzvoYg4LSJW72SdHSJiUUQ832TZahFxTkTMiIi5EXFDRIxd/qOTJEmSBobX9eXOImI0cCvwAHAAsDlwDsUHjlM6WfXQsu5ZwCPAdsAZ5f17muwngAuAmTQ/xvOB9wInlHUmALdExLaZuaAHhyZJkiQNKH0a9IHjgGHAwZk5hyJcjwQmRMTXy7JmvpaZ1TPzkyJiAXBRRGyamU821D8CWBe4FDimuiAiNgKOAo7MzCvLsj8BT5TrfW/5DlGSJEnqf309dGdf4OaGQP9jivC/a0crNYT8dn8o7zeoFkbECIoz/58GXmmy3l7l/cTK9qcDd5XtkyRJkga9vg7644Cp1YLMfAqYVy7rjp2BxcBjDeVfBB7MzGs7acMzmflSQ/mDPWiDJEmSNCD19dCd0cDsJuWzymUtiYj1KMb0/yAzZ1TKtwSOB3Zc0W2QJEmSBrJBN71mRKwKXA28RHExbdW3gMsz88+9vM9jImJKREyZOXNmb25akiRJWiH6OujPAtZsUj66XNapcjadK4Gtgf0yc1Zl2b7AO4BzI2JURIwCVi9XGxURq/W0DZl5cWa2ZWbb2muv3VUzJUmSpH7X10N3ptIwDj4iNgaG0zB2vwPfpJiWc8/MbKy/JfB6iuk3G80CTgW+XO5n44hYIzPnVuosc/2AJEmSNFj1ddC/CfhMRIzIzBfLskOB+cDkzlaMiJOAjwLvy8y7mlS5BrivoWw8cBDFh4MnyrJflvcHAVeV294AeCfwke4cjCRJkjRQ9XXQvxD4ODAxIs4CNqP4sapzq1NuRsSjwOTMPKp8/H7gTOByYHpE7FTZ5mOZOTMznwGeqe4sInYDFmbmpPayzHwmIr4PfLMcCtT+g1lPUgZ/SZIkabDr06CfmbMiYneKX629nmL2m/MognZju4ZUHrfPfT++vFV9mOIDQHd8HJgLnEsxbGgycLi/iitJkqS6iMzs7zYMKm1tbTllypT+boYkSZJqLiLuzcy2nq4/6KbXlCRJktQ1g74kSZJUQwZ9SZIkqYYM+pIkSVINGfQlSZKkGjLoS5IkSTVk0JckSZJqyKAvSZIk1ZBBX5IkSaohg74kSZJUQwZ9SZIkqYYM+pIkSVINGfQlSZKkGjLoS5IkSTVk0JckSZJqyKAvSZIk1ZBBX5IkSaohg74kSZJUQwZ9SZIkqYYM+pIkSVINGfQlSZKkGjLoS5IkSTVk0JckSZJqyKAvSZIk1ZBBX5IkSaohg74kSZJUQwZ9SZIkqYYM+pIkSVINGfQlSZKkGjLoS5IkSTVk0JckSZJqyKAvSZIk1ZBBX5IkSaohg74kSZJUQwZ9SZIkqYYM+pIkSVINGfQlSZKkGjLoS5IkSTVk0JckSZJqyKAvSZIk1ZBBX5IkSaohg74kSZJUQwZ9SZIkqYYM+pIkSVINGfQlSZKkGorM7O82DCoRMRN4sp92vxbwfD/tW/3Hfl952fcrL/t+5WXfr7ya9f2mmbl2Tzdo0B9EImJKZrb1dzvUt+z3lZd9v/Ky71de9v3Ka0X0vUN3JEmSpBoy6EuSJEk1ZNAfXC7u7waoX9jvKy/7fuVl36+87PuVV6/3vWP0JUmSpBryjL4kSZJUQwb9AS4itoqI2yJiXkQ8GxGnR8SQ/m6Xek9EjI+IbHI7rlInIuILEfF0RMyPiDsjYvv+bLe6LyK2iIiLIuJPEbEoIiY1qdNSX/u3YfBosd+nNfkb8FyTevb7IBIRh0TE/0TE9Ih4KSLujYjDm9Q7OiIeiYgFZZ3dm9TZMCL+OyJejIjnI+KCiBjeN0ei7mil3yNiUgf/96/eUG+5+v11vXVQ6n0RMRq4FXgAOADYHDiH4gPaKf3YNK0Y7wLmVx4/Xvn354FTgc8AU4ETgVsjYpvMXCYMaMDaGtgP+C0wtIM6Xfa1fxsGnVb6HeBHwLcrj1+pLrTfB6UTgSeAEyjmR98P+FFErJWZ3wYoA+CFwATgLuDDwM8j4m2Z+ZeyzlDgZorXxGHAKODc8v6IvjwgtaTLfi/dAXyhYd2X2//RK/2emd4G6A04CZgFjKyUfRaYVy3zNrhvwHgggdd3sHx14AXgi5WyNYCZwJf7u/3eutXXq1T+fQ0wqSd97d+GwXXrqt/L8mnAN7rYjv0+yG7AWk3KfgQ8UXn8EHBp9fUC/Bm4qlJ2OLAIeGOl7H3AYuBN/X2c3nrU75OAa7rYznL3u0N3BrZ9gZszc06l7MfAMGDX/mmS+sH/AUYCV7cXZOZc4HqK14gGicxc3EWVVvvavw2DSAv93ir7fZDJzGa/cPsHYAOAiNgM+CeWfs8vBn7Ksu/5ezLziUrZtRRnevfp5WZrOXXV792w3P1u0B/YxlF8db9EZj5FcfZmXL+0SCvSYxHxakQ8FBHHVsrHUXyif6Sh/oP4OqibVvvavw31dFREvBIRL0TENRGxacNy+70edgYeLv/d3m9TG+o8CIyJiLUr9Rr7/hXgMez7waLa7+32Kq+3mRcRN0fEdg3Ll7vfHaM/sI0GZjcpn1UuUz38lWJM9t3AEIpxeBdGxPDMPI+ir1/KzEUN680ChkfEquUbX4Nfq33t34b6uY5iDP8zwJuB04BfRcS2mflCWcd+H+TKi2wPBI4si9r7rbFfZ1WWz8S+H9Sa9DvAZOAK4FFgU+Bkivf8P2fmtLLOcve7QV/qZ5l5M8XFNu1uKq+6PyUivtVPzZLUhzLzE5WHv4qI3wD3UVyY+c3+aZV6U0SMpRinfV1mXt6vjVGf6ajfM/O0SrVfRcStFGfvP1neeoVDdwa2WcCaTcpH89qnfdXTNcAYYCxFX7++yRR6o4F5ns2vlVb72r8NNZfFbCsPAW+pFNvvg1REjAFuAp4EPlBZ1N5vjf06umG5fT8IddLvy8hiVrVf08vveYP+wDaVhjFYEbExMJxlx/OpXrJyP5ViSM8WDXWWGbunQa/VvvZvw8ohee1vAdjvg1I55/nPgVWBf8/MeZXF7f3WON56HPCPzJxZqdfY96sCm2HfD0hd9HtHWnnPd6vfDfoD203A3hExolJ2KMVc65P7p0nqI++lmHv3SeA3wBzgkPaF5R+Q/SleI6qPVvvavw01FxHbUPwHf2+l2H4fZCLidRQz6LwJ2CczZ1SXZ+bjFBdoVt/zq5SPG9/zb2u4QPvdwGrAL1ZM69VTXfV7B+usB+zCsu/55ep3x+gPbBcCHwcmRsRZFJ/gJgDnNkyvpkEsIn5GcSHunyjO5h5a3j5eTrO2ICK+BpwaEbN47UeUVmHpH9fRAFeG9v3KhxsCIyPiveXjGzNzXot97d+GQaSrfgf+leLHb34OPEsR8E8BngIur2zKfh98vkvR958A3hARb6gs+0NmvkzRh1dFxDSKoRv/QREQ31+pew3FxZoTI+JUiuEc5wE/yszGWbrU/zrtd2BL4KsUHwaeBDah+J2MxSx9Tc7y93t//6iAty5/dGEr4HaKMzZ/Bc4AhvR3u7z1ah+fSTEWd17Zz/cCH2yoE+Wb/Zmyzq+AHfq77d663ddjee2r2cbb2O70tX8bBs+tq34HtgNuo5hdZSHwHEXA38B+H9w3ih9C6/Q9X9Y7mmL2lZeB3wO7N9nWRhRzqL8E/B34DjC8v4/RW/f7neID/43le/iVsj9/Bozr7X6PciOSJEmSasQx+pIkSVINGfQlSZKkGjLoS5IkSTVk0JckSZJqyKAvSZIk1ZBBX5IkSaohg74k1UREvC8ixjeUTYqIa/qpScuIiGMi4sD+bockrQycR1+SaqIM9Gtl5m6Vsq2AhTlAfj0zIqYA/7+dewuxqorjOP79iVkamkoYTWoaYcWEIN0wCCsoHwqsRCKKNOghjIIMDdLwWpJYYKZFUog9dKOLhChJahnkgy+So3bRisy0yzQwznij/j2stWF3PA0jnJiZ7e8Dm8Vee629//ucl/9ee+21OyJm9HQsZmZV17+nAzAzs/9PROzp6RjMzKxneOqOmVkFSFoLTAUmSYq8LaidupPrfpd0g6Sdko5J+kLSWEkjJH0k6aikvZJurXOdhyW1SDoh6UdJc2qON0vaJKlVUkc+z6P52DbgGmB6KcYZ+diDOY5WSX9K2irp2tp7zDHfIWmPpE5JGyQNl3R57tOR24yv6RuSZklaka/RJmmlpAEN+QPMzHohj+ibmVXDYmA0MBSYmesOAjfXaTsIeA1YBnQALwFvAieAjcBqYA7wnqRREdEJIGk28Fzut42UtC+W1BkRL+dzfwzsBR7I57sCGJKPzQTeBw7keAH253IMsC7vDwDuA7ZLao6IA6XYRwOLgHn5PlbmexkDrMmxLQXezn3L81OfBHYA9wPNwLPAcWB2nd/IzKzPc6JvZlYBEbFfUivQLyJ2FPWS6jUfCDweEZ/lNk3AKmB+RCzPdQeBFmASsFHSEGA+sCQiFubzbJY0CJgn6RVgGDAWmBIRX+U2n5Zi3COpA/itHGM+tqgUcz9gM3A96YFhUanpcGBiROzPbceTEvXpEbEu1wnYAFxJeugotAPTIuLvfE/nAnMlLY2I1ro/rJlZH+apO2ZmZ5+TwPbS/ne53FKn7pJcTgTOJ43y9y+23OciYCTQCvwEvCrpXkkjuhuQpKskfSjpCPAXcIr0NmBcTdMfiiT/DGIvrM9JfuED0kPP1d2N08ysL3Gib2Z29mmvSXhP5rKtqIiIou68XF6YyxZSEl5sW3P9qHzO24HDwBvAYUnbJU3oKhhJg4FPgFHALOAm4DpgV+n6hbaa/dNiL9XV9v31P/Yv7io+M7O+ylN3zMysO4qpLXcCR+oc/xogIvYBUyWdQ0rYnwc2SBpZ83BRNpH0RuC23B8ASRc0Kvis9g1Dsf9Lg69jZtYreETfzKw6TnL6KHajfAkcA5oiYmedrb3cOCJORcQW4EXSiPnQLmIcmMsTRYWkG0kf2DbSlDz/v3AP6Z52N/g6Zma9gkf0zcyqYx8pmb2LtOLOoUadOCLaJC0AVki6FPicNFg0DrglIu7OH8YuB94hrawzDHgK2FX62HUfMFnSZOAP4HvSSjhHgTWSlpFG9xcAPzcq/mww6RuDNaRVd54BVvlDXDOrKif6ZmbVsRqYQJofPwxY2HXzMxMRyyQdAp4gLVV5HPiGlNhDmpt/BJgLNJHmzW8lJfuFJaQlMt8lLbv5UESslTSN9JCwHvgWeIS0xGcjvQBcBrxFekh5HXi6wdcwM+s19O8lhs3MzKpHUgCPldb7NzOrPM/RNzMzMzOrICf6ZmZmZmYV5Kk7ZmZmZmYV5BF9MzMzM7MKcqJvZmZmZlZBTvTNzMzMzCrIib6ZmZmZWQU50TczMzMzqyAn+mZmZmZmFfQP/7k+zBWzPf4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x504 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lineplot(x='timestamp',\n",
    "             y='value',\n",
    "             data=allmetrics,\n",
    "             units='TrainingJobName',\n",
    "             estimator=None,\n",
    "             size='is_best',\n",
    "             sizes=(3, 0.5),\n",
    "             palette=['gray', 'blue'],\n",
    "             hue='is_best',).figure.suptitle('Metric evolution over each training job');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The graph above shows how the tuned hyperparameters yield superior performance, as evidenced by the slope of the best training job."
   ]
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  },
  "nteract": {
   "version": "0.15.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
